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contributions  to  rational  approximation  and  special  functions,  and  was  a 
very  generous  friend. 


PREFACE 


This  book  tells  the  story  of  four  summer  days  in  Shrivenham  in  1988,  when  88  people 
came  to  the  Second  International  Conference  on  Algorithms  for  Approximation  at  the 
Royal  Military  College  of  Science  (RMCS)  from  July  12^  to  15^.  Of  course  the  book 
only  tells  part  of  the  story.  It  cannot  easily  convey  the  good  humour  and  very  happy 
atmosphere  that  prevailed,  and  it  does  not  catalogue  the  many  friendships  that  were  made, 
the  excellent  meals  and  refreshments  that  were  enjoyed,  and  the  plethora  of  social 

activities  that  took  place. 

The  conference  was  organised  for  two  main  reasons.  Firstly,  there  were  many  requests 
for  a  Second  Shrivenham  Conference  from  those  who  attended  the  First  in  1985. 
Secondly,  it  had  clearly  been  established  at  the  First  Conference  that  there  was  a  demand 
and  need  for  a  regular  meeting  which  emphasised  the  algorithmic  aspects  and  applications 
of  approximation.  There  was  also  a  continuing  need  for  more  conferences  in  the  UK  in 
the  general  area  of  approximation. 

The  meeting  itself  was  run  under  the  auspices  of  Cranfield  Institute  of  Technology,  of 
which  RMCS  is  a  faculty,  and  financial  support  was  gratefully  received  from  the  US  Office 
of  Aerospace  Research  and  Development.  The  organising  committee  was  Maurice  Cox 

(NPL)  and  Anne  Daman  and  John  Mason  (RMCS),  and  the  RMCS  local  organisers  were 
very  ably  assisted  by  Elizabeth  Smith  and  Pamela  Moore.  It  was  a  loss  to  RMCS  and  to 
the  conference  organisation  Mien  Anne  Daman  left  early  in  1988  to  seek  her  fortune  in 

the  USA,  but  there  were  celebrations  Mien  she  returned  in  July  as  Mrs  Anne  Trefethen 

to  receive  her  PhD  degree  and  attend  the  conference  with  her  husband. 

The  ten  invited  speakers,  chosen  by  the  organising  committee,  covered  a  broad  spectrum 
of  topics  and  came  from  a  wide  range  of  countries.  All  of  their  papers  appear  in  these 
proceedings.  The  UK  speakers  were  Dr  John  Gregory  (Brunei),  Professor  John  Mason 
(RMCS),  Professor  Michael  Powell  (Cambridge)  and  Professor  Alastair  Watson  (Dundee). 
Also  from  Europe  came  Professor  Wolfgang  Dahmen  (Berlin)  and  Professor  Tom  Lyche 
(Oslo).  The  speakers  from  the  USA  were  Dr  Eric  Grosse  (Bell  Labs),  Professor 
Larry  Schumaker  (then  Texas  A  and  M,  now  Vanderbilt)  and  Professor  Lloyd  Trefethen 
(MIT).  Finally  from  Canada  came  the  opening  speaker,  Professor  Ian  Barrodale 
(Barrodale  Associates,  Victoria),  who  set  the  tone  of  all  the  invited  talks  with  a  paper  of 
excellent  content  and  very  entertaining  delivery.  It  is  also  appropriate  at  this  point  to 
mention  Dr  James  Lyness  (Argonne),  who  as  guest  speaker  at  the  conference  dinner, 
added  further  academic  distinction  and  good  humour  to  the  occasion. 


The  remainder  of  the  conference  programme  was  made  up  of  submitted  papers,  of  which 
forty  were  selected  for  two  parallel  sessions.  Within  this  book  you  will  find  the 

thirty-three  of  these  papers  which  were  Finally  offered  and  accepted  for  publication  after  a 
refereeing  procedure.  There  was  general  agreement  on  the  high  standard  of  the 
conference  talks,  and  we  believe  that  this  is  reflected  in  the  overall  quality  of  the 
published  papers. 

It  will  be  noted  that  the  last  paper  in  the  book,  by  Eric  Grosse,  is  a  substantial  catalogue 
of  algorithms  for  approximation,  and  we  believe  that  this  will  prove  to  be  a  very  useful 
and  popular  reference.  Moreover  we  should  not  be  suprised  to  see  this  becoming  the 

catalyst  for  a  series  of  such  offerings.  Unlike  the  proceedings  of  the  First  conference,  the 
present  volume  does  not  include  a  "software  panel".  However,  this  is  more  than 
compensated  for  by  the  inclusion  of  the  above-mentioned  catalogue  and  the  separate 
publication  (by  Chapman  and  Hall)  of  a  volume  of  proceedings  of  a  Symposium  on 
"ScientiFic  Software  Systems",  held  at  RMCS  on  the  day  before  the  conference  (July  11th). 
The  latter  volume  of  seventeen  papers  covers  the  broad  area  of  numerical  software,  as 

well  as  a  wide  range  of  software  and  hardware  requirements.  It  includes,  amongst  many 

contributions  which  might  interest  current  readers,  two  papers  with  immediate  application 
to  approximation,  viz  "Linear  algebra  support  modules  for  approximation  and  other 
software"  by  M  G  Cox  and  "An  advisory  expert  system  for  curve  and  surface  Fitting"  by 
Anne  E  Trefethen. 

The  forty-one  papers  in  the  present  volume  have  been  arranged  into  three  primary 

sections:  I  Development  of  Algorithms,  n  Applications,  and  HI  Catalogue  of  Algorithms. 
The  First  two  sections  have  been  subdivided  into  eight  groups:  1.  Spline  approximation, 
2.  Polynomial  and  piecewise  polynomial  approximation,  3.  Interpolation,  4.  Smoothing  and 
constraint  methods,  S.  Complex  approximation,  6.  Computer-aided  design  and  geometric 
modelling,  7.  Applications  in  numerical  analysis,  and  8.  Applications  in  other  disciplines. 

Such  a  division  into  sections,  while  giving  the  book  a  useful  structure,  is  somewhat 
arbitrary,  and  we  apologise  to  any  authors  who  may  feel  that  their  work  has  been 
incorrectly  categorised.  Several  papers  could  have  been  placed  in  up  to  three  groups 
(especially  spline  approximation,  piecewise  polynomial  approximation,  and  computer-aided 
design).  Moreover  the  CAD  group,  which  we  have  placed  in  the  Applications  section 

could  perfectly  well  have  been  placed  in  Section  I.  Although  there  is  no  group  headed 

"nonlinear  approximation",  there  are  several  nonlinear  algorithms  (in  Section  n  in 

particular),  and  of  course  the  complex  algorithms  (in  group  5  and  elsewhere)  could  have 
come  under  this  heading. 


We  must  conclude  with  some  essential  but  broad  expressions  of  our  gratitude.  Firstly,  we 


thank  the  multitude  of  staff  at  Royal  Military  College  of  Science,  National  Physical 
Laboratory,  US  Air  Force  European  Office  of  Aerospace  Research,  and  Chapman  and  Hall 
(publishers)  who  contributed  in  so  many  different  ways  to  make  the  conference  a  great 
success  and  to  print  the  abstracts  and  proceedings.  Secondly,  we  thank  all  the  speakers 
and  authors  of  invited  and  contributed  papers,  without  whose  industry  and  patience  this 
volume  would  not  have  existed. 

As  soon  as  conference  number  II  was  over,  we  were  tempted  to  start  thinking  in  terms  of 
a  conference  series.  We  certainly  intend  that  there  should  be  further  conferences  on 
"Algorithms  for  Approximation",  since  the  first  two  generated  so  much  good  research  and 
goodwill.  All  that  we  need  are  energy,  time  and  resources  and,  of  course,  your  support. 


John  Mason 
Shrivenham 


Maurice  Cox 
Teddington 
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DEVELOPMENT  OF  ALGORITHMS 


1.  Spline  Approximation 


CONSTRAINED  SPLINE  APPROXIMATION  OF  FUNCTIONS  AND  DATA 
BASED  ON  CONSTRAINED  KNOT  REMOVAL 


( 


ERLEND  ARGE,  MORTEN  DjEHLEN,  TOM  LYCHE,  KNUT  M0RKEN 
Institute  of  Informatics 
University  of  Oslo 


Abstract  Two  of  the  authors  (Lyche  and  M0rken)  have  recently  developed  a 
knot  removal  strategy  for  splines  in  B-spline  format,  which  has  been  applied  suc¬ 
cessfully  to  the  approximation  of  functions  and  data.  In  this  paper  we  show  how 
general  constraints  can  be  incorporated  in  this  strategy  and  also  in  approxima¬ 
tion  methods  based  on  knot  removal.  In  our  implementation  of  the  knot  removal 
strategy,  two  fundamental  properties  of  B-splines  were  central — knot  insertion  or 
subdivision  and  the  fact  that  the  usual  L?  -norms  for  splines  can  be  approximated 
well  by  some  simple  discrete  norms.  Together  with  the  fact  that  a  B-spline  ex¬ 
pansion  is  a  convex  combination  of  the  coefficients,  these  properties  are  also  the 
key  features  in  our  treatment  of  constraints. 

Key  words:  Splines,  B-splines,  Constrained  approximation,  Knot  insertion, 
Knot  removal,  Discrete  norms,  Quadratic  optimization. 


1.  Introduction 

In  Lyche  and  M0rken  (1988),  a  strategy  for  removing  knots  from  a  B-spline  func¬ 
tion  without  perturbing  the  spline  more  than  a  prescribed  tolerance  was  devel¬ 
oped.  It  was  also  shown  how  this  strategy  could  be  used  successfully  to  compute 
approximations  to  large  sets  of  discrete  data,  by  applying  knot  removal  to  some 
simple  initial  approximation  like  a  cubic  spline  representation  of  the  piecewise 
linear  interpolant  to  the  data.  This  also  provides  a  method  for  approximating 
functions  as  long  as  they  can  be  sampled  sufficiently  often.  The  knot  removal 
technique  was  extended  to  parametric  curves  and  surfaces  in  Lyche  and  M0rken 
(1987b). 

Our  general  approach  to  constrained  spline  approximation  will  follow  the 
same  pattern  as  for  unconstrained  approximation.  First  we  compute  an  initial 


1  . 


approximation  satisfying  the  constraints,  but  in  general  requiring  a  large  number 
of  parameters  for  its  representation.  Then  we  remove  knots  from  this  spline,  but 
all  the  time  making  sure  that  we  do  not  violate  the  constraints  or  the  prescribed 
tolerance. 

The  literature  on  constrained  approximation  has  grown  considerably  in  the 
last  few  years,  see  Fontanella  (1987)  and  Utreras  (1987)  for  two  recent  surveys, 
and  also  the  bibliography  compiled  by  Franke  and  Schumaker  (1987).  We  will 
consider  a  general  class  of  constraints  which  requires  one  or  more  derivatives  or 
the  integral  of  the  approximation  to  be  bounded  by  general  functions. 

In  the  remaining  part  of  this  section  we  introduce  our  notation  and  give 
some  fundamental  results  concerning  splines.  In  Section  2  we  discuss  how  various 
constraints  can  be  written  in  terms  of  linear  inequality  and  equality  constraints 
involving  the  B-spline  coefficients  on  a  suffiently  large  knot  vector.  In  order  to 
carry  out  constrained  knot  removal,  it  is  desirable  to  have  an  initial  approximation 
which  satisfies  the  constraints.  The  initial  approximation  scheme  should  also  cope 
with  the  situation  where  the  data  do  not  satisfy  the  constraints.  We  discuss  these 
questions  in  Section  3.  Knot  removal  is  the  topic  of  Section  4.  We  review  the 
unconstrained  knot  removal  strategy  in  Lyche  and  Morken  (1988)  and  explain 
how  it  can  be  extended  to  handle  constraints.  The  paper  ends  with  a  section 
discussing  examples  of  constrained  approximation  using  the  proposed  technique. 

1.1  Notation 

All  splines  in  this  paper  are  represented  in  terms  of  B-splines.  We  denote  the  i  ’th 
B-spline  of  order  k  on  the  knot  vector  t  by  #»,*,»  and  assume  the  B-splines  to 
be  normalized  to  sum  to  one.  If  t  contains  m  +  k  elements  with  none  occurring 
more  than  k  times,  then  we  can  form  m  linearly  independent  B-splines  on  t . 
These  B-splines  span  a  linear  space  of  splines  defined  by 

m 

Sfc.t  =  1 5^  diBi,k,t  |  di  €  R  for  i  =  1 ,  2,  . . . ,  m  }. 

i=i 

In  this  paper  we  will  assume  that  m  >  k,  and  that  t*  <  t*+i  and  tm  <  tm+i , 
and  we  will  only  be  interested  in  the  spline  functions  on  the  interval  [<t,tm+i]. 

1.2  Knot  insertion 

The  central  ideas  of  this  paper  are  consequences  of  the  fact  that  the  B-spline 
coefficients  can  model  the  spline  they  represent  with  arbitrary  precision.  This  is 
based  on  so  called  subdivision  or  knot  insertion  techniques,  so  let  us  consider  the 
basis  for  this.  A  spline  on  a  knot  vector  r  can  also  be  represented  as  a  spline  on 
any  knot  vector  t  that  contains  all  the  knots  of  r .  The  reason  for  this  is  that  if 
r  is  a  subsequence  of  t ,  then  Sk,r  is  a  subspace  of  Sk,f .  Let  g  be  a  spline  in  S*,r 
with  coefficient  vector  c  relative  to  r  and  coefficient  vector  b  relative  to  t .  Then 
c  and  b  are  related  by  the  equation  b=  Ac,  where  A  is  the  knot  insertion  matrix 


of  order  k  from  r  to  t.  This  matrix  has  properties  very  similar  to  the  B-spline 
collocation  matrix,  see  Jia  (1983),  Lyche  and  M0rken  (1987a,  1988),  and  Lyche 
(1988).  One  consequence  of  these  properties  is  that  as  more  and  more  knots  are 
inserted,  the  B-spline  coefficients  converge  to  the  given  spline  g ,  see  Cohen 
Schumaker  (1985)  and  Dahmen  (1986),  where  it  is  shown  that  the  convergence  is 
quadratic  in  the  knot  spacing.  This  means  that  if  we  work  on  a  sufficiently  fine 
knot  vector,  we  do  not  lose  much  by  using  the  coefficients  as  a  representation  of 
the  spline. 


1.3  Discrete  norms 

The  ability  of  the  B-spline  coefficients  to  mimic  the  spline  they  represent  is 
exemplified  by  the  fact  that  certain  simple  combinations  of  the  B-spline  coeffi¬ 
cients  provide  good  approximations  to  the  Lp  -norms  of  the  spline.  Specifically, 
if  /  =  diBi'k't  is  a  spline  on  a  knot  vector  of  length  m  +  k  as  above,  we 
can  define  a  family  of  discrete  norms  called  the  (£?,t)-norms  on  S*.  t  by 


,,  =  { 


{£,=1  !<*•  !'(*»+*  -  *i)/*}1,F»  for  1  <  p  <  oo; 


max,  |d;|, 


for  p  =  oo. 


Equivalently,  if  we  define  the  diagonal  m  x  m  matrix  EXJV  with  the  i  ’th  diagonal 
element  given  by 


(E\,v)i  i  =  1  “  *<)/*} ^  for  1  <  p  <  oo; 

1,  for  p  =  oo; 

then  we  also  have 

ll/ll*,.  = 

Here  ||  -  ||p  denotes  either  the  usual  Lp  norm  of  a  function  or  the  £p  norm  of  a 
vector.  The  significance  of  the  (£p,  t) -norms  is  due  to  the  fundamental  inequalities 

D;'\\!\W.r  <  I/I,  <  ||/||„.t  <  ii/H, 

where  rCt  and  /  €  StiT .  The  leftmost  inequality  is  due  to  de  Boor  (1976a  and 
1976b).  The  number  Dk  depends  only  on  k,  and  numerical  experiments  have 
shown  that  D*  ~  2* .  As  an  example  we  have  Z?4  «  10. 

The  (£*,t)-norm  is  of  course  also  a  norm  on  any  subspace  of  S*t< .  If  a 
spline  in  a  subspace  of  S*,<  is  given,  then  to  compute  its  (£*,*) -norm  it  must  be 
represented  as  a  spline  in  t  by  knot  insertion  or  degree  elevation. 

We  will  be  using  the  (£°°,t)-norm  to  estimate  the  relative  importance  of 
the  interior  knots  during  knot  removal,  and  the  error  of  an  approximation  will 
be  measured  in  this  norm  since  it  gives  an  upper  bound  on  the  L°°  -norm.  The 
method  we  will  employ  to  compute  spline  approximations  will  be  best  approxi¬ 
mation  in  the  (£*,i)-norm.  This  norm  converges  to  the  L2-norm  for  continuous 


functions  as  the  knot  spacing  goes  to  zero,  see  Lyche  and  Morken  (1988),  and  it 
can  even  be  shown  that  the  best  approximation  in  the  norm  converges  to 

the  best  approximation  in  the  L2  -norm.  The  main  advantage  of  working  with 
this  discrete  norm  instead  of  the  L2  -norm  is  computational  efficiency. 

Compared  with  other  ((?,  t)-norms,  the  (^2,  £)-norm  has  the  advantage  that 
best  approximation  leads  to  a  simple  linear  system  of  equations.  When  it  comes 
to  constrained  approximation  this  may  not  be  so  crucial,  and  approximation  in 
the  (£*,t) -norm  and  the  (£°°,£)-norm  sue  interesting  alternatives.  To  develop 
efficient  algorithms  for  constrained  spline  approximation  in  these  norms  will  be 
the  subject  of  future  work  by  the  authors. 


2.  Constraints 


Our  approach  to  constrained  approximation  is  also  based  on  the  fact  that  the 
B-spline  coefficients  model  the  spline  they  represent.  Let  /  =  diB,k,t  be 
a  spline  in  $ k,t  to  be  approximated  by  a  spline  g  =  53"=1  c.B.tr  in  a  subspace 
§k,r ,  and  denote  by  b  the  coefficient  vector  of  g  relative  to  t,  so  that  b  =  Ac. 
We  will  consider  constraints  of  the  form 

E\  b  >  t>i , 

(1) 

E2b  —  t>2, 


where  E\  and  E2  are  rectangular  matrices  and  Vj  and  v2  are  vectors.  (The 
notation  u  >  w  for  vectors  u  and  w  denotes  the  component-wise  inequalities 
Ui  >  u>j  for  »'  =  1,  2,  ....)  In  other  words,  the  constraints  can  be  expressed 
directly  as  restrictions  on  linear  combinations  of  the  coefficients  relative  to  the  t 
knot  vector.  The  constraints  characterize  the  set  of  permissible  spline  functions, 

m 

F*,<  ={’/’  =  |  Eib  -  Vi  &  ^ b  =  ”2}- 

1=1 


As  mentioned  above,  we  will  use  best  approximation  in  the  (£■*,£) -norm  to  com¬ 
pute  spline  approximations.  The  typical  optimization  problem  to  be  solved  is 
therefore 


min 


11/  -  V’IUm, 


(2) 


and  a  matrix  formulation  of  this  problem  is  given  in  Section  4.2.  This  problem 
is  a  quadratic  minimization  problem  with  linear  constraints.  Such  problems  are 
studied  extensively  in  the  optimization  literature,  see  e.g.  Fletcher  (1987).  We 
emphasize  that  even  though  the  approximation  g  is  a  spline  on  the  knot  vector 
r,  we  minimize  a  weighted  £2-norm  of  the  B-spline  coefficients  of  the  error  on 
the  t  knot  vector  which  for  good  results  should  contain  ‘many’  knots. 

In  practice,  any  constraints  that  can  be  handled  by  the  quadratic  minimiza¬ 
tion  routine  that  is  used  to  solve  (2)  are  admissible,  but  in  this  paper  we  restrict 


our  attention  to  linear  constraints  of  the  type  (1).  It  should  be  noted  here  that 
constrained  problems  for  parametric  curves  and  surfaces  often  lead  to  nonlinear 
constraints,  cf.  Ferguson,  Frank  and  Jones  (1988). 

It  has  been  observed  that  bounds  on  a  spline  g  or  its  derivatives  can  be 
replaced  by  a  finite  number  of  linear  inequality  constraints  on  the  B-spline  coeffi¬ 
cients  of  g,  see  e.g.  Cox  (1987).  In  general  though,  the  replacing  constraints  will 
be  stronger  than  the  original  ones  so  that  the  set  of  feasible  solutions  is  overly 
restricted.  Recall  that  during  knot  removal  we  also  have  a  requirement  that  the 
error  should  be  less  than  the  tolerance.  This  additional  constraint  together  with 
the  constraint  on  the  B-spline  coefficients  may  result  in  a  problem  having  no 
feasible  solutions.  To  weaken  the  constraints,  we  propose  to  use  knot  insertion 
and  constrain  the  B-spline  coefficients  on  a  knot  vector  which  has  many  knots 
compared  to  the  number  of  oscillations  in  the  spline  to  be  approximated.  In  this 
way  the  discrepancy  between  the  original  and  replacing  constraints  is  reduced. 

As  an  example,  consider  the  polynomial  T(x)  =  4x3  —  3x  + 1  on  the  interval 
[—1, 1]  as  a  given  function  to  be  approximated  on  the  Bezier  knot  vector  r  with 
four  knots  at  —1  and  1 ,  with  L°°  -error  less  than  c  and  with  nonnegativity  as  the 
constraint.  The  function  T  is  the  cubic  Chebyshev  polynomial  with  the  constant 
1  added  to  make  it  nonnegative.  If  we  choose  the  approximation  g  to  be  T, 
then  we  solve  the  problem  exactly.  However,  a  general  purpose  algorithm  for 
solving  this  type  of  problem  would  probably  not  discover  that  T  is  nonnegative 
and  a  feasible  solution.  Our  approach  implements  the  ideas  above.  A  sufficient 
condition  for  nonnegativity  is  that  the  B-spline  coefficients  on  some  knot  vector 
are  nonnegative,  and  instead  of  requiring  the  X- 00 -error  to  be  less  than  e  we 
require  the  (t°°,  t) -error  to  be  less  than  e  for  a  suitable  t.  Note  that  on  the 
Bezier  knot  vector  r,  one  of  the  B-spline  coefficients  equals  —4  so  this  knot 
vector  is  not  suitable  as  t,  since  the  nonnegativity  constraint  would  lead  to  a 
(£°°,t)-error  of  at  least  4.  If  we  insert  the  2q  —  1  interior  knots  (— 1  +  1  /q,  —1  + 
2/q, ...  ,—1/q,  0, 1/q,  2/q, . . . ,  1  —  1  /q)  into  r ,  where  q  is  a  positive  integer,  then 
it  is  easily  seen  that  on  this  refined  knot  vector  t9 ,  all  the  B-spline  coefficients  of 
T  are  greater  than  —2/q2 .  Therefore,  for  the  set  of  feasible  solutions 


F4 ,i»  =  {</>  =  llr  ~  <  e  &  Ci  >  0  for  all  i  j 


to  be  nonempty,  we  must  have  2/q2  <  t  or  q  >  y/2/ yfk. 

In  the  rest  of  this  section  we  assume  that  we  have  a  knot  vector  t  on  which  the 
original  constraints  have  been  replaced  by  constraints  on  the  B-spline  coefficients 
in  a  satisfactory  way.  We  now  discuss  some  specific  constraints,  and  we  start  by 
considering  nonnegativity  constraints  in  general.  As  was  indicated  in  the  example, 
the  obvious  way  to  implement  straightforward  nonnegativity  is  to  require  the  B- 
spline  coefficients  on  t,  of  an  approximation  g  =  ,  to  be  nonnegative, 


K  >0, 


for  t  =  1 ,  2 ,  . . . ,  m . 


In  many  cases  one  only  wants  nonnegativity  locally,  say  on  an  interval  [a,  6] 
which  is  contained  in  [t*,  tm+i] .  This  is  accomplished  by  requiring  nonnegativity 
of  only  the  coefficients  that  multiply  a  B-spline  with  part  of  its  support  in  [a,  6] . 
To  be  able  to  strictly  enforce  such  conditions,  it  may  be  necessary  to  refine  t  by 
including  a  and  b  as  knots  of  multiplicity  k  —  1 . 

A  simple  generalization  of  the  nonnegativity  constraint  is  to  require 

bi  >  e,  for  i  =  1 ,  2 ,  . . . ,  m , 

for  some  real  number  e.  The  next  step  is  then  to  require  g  >  h  for  a  spline 
h  =  on  a  knot  vector  t\.  To  be  able  to  compare  h ,  /  and  an 

approximation  g  we  then  replace  t  by  t  U  t\  and  assume  that  on  the  new  t  knot 
vector  the  constraining  spline  is  given  by  h  =  e.,Bi  k,t  •  Then  we  would 

simply  require 

bi  >  ei ,  for  i  =  1 ,  2 ,  . . . ,  m . 

These  more  general  constraints  are  easily  restricted  to  some  interval  just  as  above. 

All  of  these  constraints  have  restricted  the  approximation  from  below.  If  a 
restriction  from  above  is  required,  this  is  most  conveniently  expressed  as  —b,  > 
-ei. 

There  are  many  other  types  of  constraints  that  can  be  written  as  linear 
equality  and  inequality  constraints. 

•  Monotonicity  and  convexity. 

A  monotone  approximation  g  is  characterized  by  g'  <  0  or  g'  >  0  and  a 
convex  approximation  by  g"  >0.  By  the  well  known  differentiation  formula 
for  B-splines,  these  constraints  can  be  implemented  similarly  to  the  nonneg¬ 
ativity  constraints  and  also  generalized  in  the  same  way.  In  general,  any 
derivative  of  order  less  than  k  can  be  constrained  in  this  manner. 

•  Interpolation. 

The  approximation  can  be  required  to  interpolate  a  given  value  y  at  a  point 
x  leading  to  an  equality  constraint  of  the  form, 

m 

5 ZbiBi,kAx )  =  y- 

i=i 

It  is  well  known  that  at  most  k  B-splines  are  nonzero  at  x  so  that  this  sum 
contains  at  most  k  terms.  Any  derivative  can  of  course  be  interpolated  in  a 
similar  way. 

•  Integral  constraints. 

The  integral  of  the  approximation  can  be  constrained  by  requiring 


The  equality  follows  because  the  integral  of  B,  k  t  is  known  to  be  (t;+i  — 
ti)/k.  If  only  the  integral  over  a  small  interval  [a,  6]  is  to  be  constrained, 
one  could  insert  k  —  1 -tuple  knots  at  a  and  b. 

•  Smoothness. 

An  approximation  g  with  a  required  smoothness  is  obtained  by  restricting 
the  multiplicity  of  the  knot  vector  on  which  g  is  defined. 

•  Discontinuities. 

Jumps  in  derivatives  can  be  enforced  by  using  some  of  the  above  techniques. 
If  we  want  a  sharp  edge  at  the  point  a  for  instance,  we  must  first  ensure 
that  a  occurs  with  multiplicity  k  —  1  in  the  knot  vector.  We  can  then  for 
example  enforce  g'(x)  >  1  for  x  <  a  and  g'(x)  <  0  for  x  >  a.  In  fact,  it  is 
easily  seen  that  it  is  sufficient  to  restrict  only  three  coefficients. 

Any  linear  combination  of  the  constraints  above  is  also  a  valid  constraint.  As  an 
example  one  could  force  the  approximation  to  take  on  the  value  of  its  integral  at 
a  given  point  by  combining  interpolation  and  integration  constraints. 

It  should  be  noted  that  the  requirement  that  the  (£°°,t) -error  should  be 
less  than  the  tolerance  can  be  implemented  as  a  constraint  of  the  above  type. 
However,  it  seems  more  efficient  to  implement  this  constraint  in  the  same  manner 
as  in  unconstrained  knot  removal,  cf.  Section  4. 

3.  The  initial  approximation 

In  general,  constrained  approximation  of  data  by  constrained  knot  removal,  will 
require  two  steps  as  described  in  the  introduction.  First,  an  initial  spline  approx¬ 
imation  to  the  data  satisfying  the  constraints  is  computed,  and  then  constrained 
knot  removal  is  applied  to  this  initial  approximation.  The  purpose  of  the  second 
step  is  to  remove  those  knots  in  the  initial  approximation  that  are  redundant 
relative  to  a  given  tolerance.  This  step  will  be  described  in  Section  4.  Here  we 
will  discuss  the  problem  of  computing  an  initial  approximation. 

In  general  we  also  determine  an  initial  approximation  in  two  stages.  First,  the 
data  are  converted  to  a  spline  <j>  which  does  not  necessarily  satisfy  the  constraints. 
Then,  on  the  knot  vector  t  of  </>,  we  compute  a  spline  approximation  f  to  <f> 
satisfying  the  constraints. 

Let  us  discuss  each  stage  in  more  detail.  For  the  first  stage  we  are  given 
some  data  to  be  approximated.  These  data  can  be  in  different  forms.  It  may 
be  a  set  of  discrete  data,  a  general  function  or  a  spline  function.  The  purpose 
of  Stage  1  is  to  convert  the  data  into  a  spline  4>  of  order  k  on  a  knot  vector  t. 
If  the  data  is  a  set  of  discrete  points  (ij,  y,)  with  the  abscissae  increasing,  we 
can  let  4>  be  either  the  linear  interpolant  to  the  data,  the  Schoenberg  variation 
diminishing  spline,  or  we  can  determine  4>  by  using  unconstrained  knot  removal 
or  some  other  suitable  method.  Similarly,  if  the  data  is  a  general  function  or  a 
spline  of  order  greater  than  k ,  we  can  approximate  it  by  a  spline  of  order  k  by 


using  the  same  method.  In  either  case  we  end  up  with  a  spline  function  4>  of 
order  k  on  a  knot  vector  t  of  length  m  +  k  with  B-spline  coefficients  (a,)"Lj . 

In  constrained  approximation  there  will  in  general  be  a  conflict  between 
satisfying  the  constraints  and  making  the  error  smaller  than  the  tolerance,  since 
the  original  data  may  not  satisfy  the  constraints.  As  an  example  consider  the 
case  where  the  original  data  take  on  the  value  —1  at  a  point,  but  the  constraints 
require  the  approximation  to  be  nonnegative  everywhere  and  the  tolerance  is  0.01 . 
To  cope  with  situations  of  this  kind,  we  need  Stage  2  to  adjust  the  spline  4>  above 
to  a  spline  /  on  the  same  knot  vector  that  satisfies  the  constraints.  This  is  done 
by  simply  minimizing  the  £2,  t-norm  of  the  error  and  enforcing  the  constraints. 
In  other  words,  the  spline  /  solves  the  quadratic  optimization  problem 

nun  ||4  -  (3) 

0€F»,, 

which  is  just  a  special  case  of  (2).  This  is  equivalent  to  the  problem 

min  || E\'2{z  -  a)||2 

subject  to 

Exz  >  vt, 

E^z  =  t>2- 

The  attitude  taken  here  is  that  it  is  more  important  to  satisfy  the  constraints 
than  to  keep  the  error  smaller  than  the  tolerance.  This  spline  will  then  be  given 
as  input  to  the  knot  removal  process. 

It  should  be  noted  that  this  step  is  not  always  necessary.  Consider  once  again 
a  discrete  set  of  nonnegative  data.  If  we  take  the  piecewise  linear  interpolant  to 
the  data  as  <f> ,  then  this  spline  will  automatically  satisfy  the  constraints  and 
we  can  set  f  =  <f>.  The  same  applies  with  global  monotonicity  and  convexity 
constraints  if  we  approximate  the  discrete  data  by  piecewise  linear  interpolation 
or  by  the  Schoenberg  variation  diminishing  spline. 

In  some  situations  the  minimization  problem  (2)  will  succeed  in  finding  a  g 
in  F*,*  even  if  /  £  F*,t.  This  would  for  example  be  the  case  when  <f>  >  — 10~8, 
the  tolerance  is  0.01  and  the  constraint  is  nonnegativity. 


4.  Knot  removal 

With  the  initial  approximation  /,  the  tolerance,  and  the  constraints  given,  we 
can  start  to  remove  knots.  More  precisely  we  have  the  following  problem. 

Constrained  knot  removal  problem.  Given  a  polynomial  order  k,  a  knot 
vector  t,  a  set  of  linear  constraints  characterizing  a  subset  F*,i  of  S*>f ,  a  spline 
/  in  Sk,t  and  a  tolerance  e;  find  a  knot  vector  rCt  which  is  as  short  as  possible 
and  a  spline  g  €  S*,r  H  Ftt<  such  that  |j/  —  <  e. 

To  discuss  this  problem  we  first  review  unconstrained  knot  removal. 


4.1  Review  of  unconstrained  knot  removal 

In  knot  removal  one  is  given  a  spline  /  in  S*>t  and  a  tolerance  e;  the  goal  is  to 
determine  a  spline  g  in  a  subspace  St,r  of  S of  lowest  possible  dimension,  such 
that  ||/  —  <7 1|  <6.  The  norm  used  here  is  in  principle  arbitrary,  but  in  Lyche  and 
M0rken  (1988)  we  used  the  (£°°,f)-norm. 

Since  finding  the  shortest  possible  r  seems  very  difficult,  the  method  only 
attempts  to  find  an  approximate  solution.  The  idea  is  to  compute  a  ranking  of 
the  interior  knots  of  t  according  to  their  significance  in  the  representation  of  / , 
and  then  try  to  remove  as  many  knots  as  possible  according  to  this  ranking.  More 
specifically,  for  each  interior  knot  ti  its  weight  w,  is  computed  as  the  absolute 
error  in  best  (£°°,t)  approximation  to  /  from  the  space  *. }  •  Clearly,  if  t, 
is  redundant  then  tv,  =  0.  (This  description  is  correct  if  all  the  interior  knots 
are  simple;  multiple  interior  knots  cause  some  complications.)  We  could  then 
rank  the  knots  according  to  their  weights  re, ,  but  this  would  not  work  very  well 
when  many  knots  have  more  or  less  equal  weights.  To  compensate  for  this,  knots 
with  similar  weights  are  grouped  together.  The  first  group  consists  of  the  knots 
with  weights  between  0  and  c/2,  the  second  group  those  with  weights  between 
c/2  and  c  and  so  on.  The  knots  in  each  group  are  listed  in  the  order  in  which 
they  occur  in  the  knot  vector.  Suppose  that  there  are  40  knots  in  the  first  group 
and  30  knots  in  the  second  group,  and  that  we  want  to  remove  50  knots.  Those 
50  knots  would  then  be  the  40  knots  in  the  first  group  plus  10  knots  from  the 
second  group.  The  10  knots  from  the  second  group  would  be  every  third  knot  in 
the  order  that  they  occur  in  the  group. 

It  is  not  possible  to  determine  from  the  weights  themselves  how  many  knots 
can  be  removed  without  the  error  exceeding  the  tolerance.  Therefore,  the  exact 
number  of  knots  to  be  removed  is  determined  by  a  binary  search.  First,  half  the 
knots  are  removed  and  an  approximation  is  computed  as  the  best  approximation 
in  the  (f^ij-norm,  together  with  the  error  in  the  (£°°,f)-norm.  If  the  error  is 
too  large,  the  approximation  is  discarded  and  we  try  to  remove  1/4  of  the  knots 
instead.  If  the  error  is  acceptable,  we  save  the  approximation  and  try  to  remove 
3/4  of  the  knots.  This  process  is  continued  until  the  exact  number  of  knots  that 
can  be  removed  has  been  determined. 

By  running  through  this  process,  a  spline  approximation  g  with  knot  vector 
r  is  determined.  However,  it  turns  out  that  it  is  usually  possible  to  remove  even 
more  knots.  The  knots  r  of  g  are  of  course  also  knots  of  /  and  now  we  can 
compute  a  ranking  of  the  knots  of  r  as  their  significance  in  the  representation 
of  g.  This  ranking  can  then  be  used  to  remove  more  knots  from  t  to  obtain  an 
approximation  to  /  with  even  fewer  knots  than  r .  The  process  terminates  when 
no  more  knots  can  be  removed  without  the  error  exceeding  the  tolerance. 

The  knot  removal  process  outlined  above  constitutes  a  convenient  basis  for 
general  spline  approximation  methods.  Since  we  have  a  method  for  removing 
knots  from  a  spline,  we  just  have  to  produce  a  good  approximation  to  a  func¬ 
tion  or  to  discrete  data  without  worrying  too  much  about  the  number  of  knots 
since  this  can  be  reduced  afterwards.  Possible  initial  approximation  schemes  in- 


elude  piecewise  linear  approximation,  cubic  Hermite  interpolation,  Schoenberg 
variation  diminishing  approximation  and  many  more. 

4.2  Constrained  knot  removal 

The  general  philosophy  used  in  unconstrained  knot  removal  can  also  be  applied 
to  the  constrained  case.  The  following  changes  in  the  strategy  used  in  the  last 
subsection  are  necessary  for  solving  the  problem  stated  at  the  beginning  of  this 
section. 

•  The  fixed  knot  approximation  method  used  during  knot  removal  has  to  pre¬ 
serve  the  constraints. 

•  The  ranking  procedure  also  includes  spline  approximation,  and  in  general 
the  constraints  must  be  taken  into  account  when  the  weights  are  computed. 

Constrained  ranking  is  currently  under  investigation  by  the  authors,  and  we  will 
not  discuss  it  any  further  in  this  paper.  In  the  examples  in  Section  5  we  will  use 
the  same  ranking  procedure  as  in  the  unconstrained  case. 

The  fixed  knot  approximation  method  was  introduced  in  Section  2.  Suppose 
that  we  have  decided  to  remove  some  knots  and  have  a  knot  vector  r  which 
is  a  subsequence  of  t.  We  want  to  find  an  approximation  g  on  r  to  /  which 
satisfies  the  constraints.  We  determine  g  by  the  optimization  problem  (2).  If 
r  has  length  n  +  k ,  then  the  B-spline  coefficients  of  g  must  solve  the  quadratic 
minimization  problem 

min  ||E -  d)||2 

i€R" 

subject  to 

E\Az  >  V|, 

E^Az  =  vj. 

The  knot  removal  process  will  continue  until  it  is  not  possible  to  remove  any  more 
knots  without  either  violating  the  tolerance  or  the  constraints. 

5.  Examples 

In  this  section  we  give  three  examples  of  the  use  of  the  constrained  approximation 
scheme  outlined  in  the  preceding  sections.  The  quadratic  optimization  problems 
involved  have  been  solved  by  general  purpose  library  software.  Since  most  of  the 
constraints  cause  the  matrices  E\  and  Ej  to  be  very  sparse,  performance  can 
be  improved  considerably  both  in  terms  of  storage  and  CPU-time  by  tailoring 
the  optimization  routines  to  this  type  of  problems.  This  we  consider  to  be  an 
important  area  of  future  research. 


i 


5.1  One-sided  approximation 

The  method  outlined  in  the  previous  sections  can  be  used  to  construct  one-sided 
approximations  to  functions.  In  this  example  a  tolerance  e  =  10-2  is  given,  and 
we  want  to  find  a  cubic  spline  approximation  g  to  the  function 

$(x)  =  max(sin(irx),0),  for  x  6  [0,4], 

subject  to  the  constraint  g(x )  —  $(x)  >  0  for  all  x  6  [0,4].  To  construct  an 
initial  cubic  spline  approximation  to  $ ,  let 

209 

/«=£(««) +io->)ij, 

t=i 

where 

t*  =  -(<i+i  +  *«+2  +  ti+ 3),  1  =  1,  2,  . . . ,  209, 

and  the  knot  vector  t  consists  of  the  numbers  (0.02  *  j)  for  j  =  0,  1 ,  . . . ,  200, 
with  knots  of  multiplicity  four  at  0  and  4 ,  and  knots  of  multiplicity  three  at  1 , 
2  and  3. 


0  12  3  4 


Figure  1.  Initial  one-sided  approximation  to  a  given  function. 

The  spline  /  is  the  Schoenberg  variation  diminishing  approximation  to  $  + 10~3 
which  satisfies  the  constraints,  but  has  far  too  many  knots.  Figure  1  shows  the 
approximation  /  (solid  curve)  and  the  error  /  —  $  (dashed  curve).  The  location 
of  the  knots  of  /  are  indicated  a.  the  bottom  of  Figure  1. 

With  the  constrained  knot  removal  procedure  described  in  the  previous  sec¬ 
tions,  an  approximation  g  to  /  is  found  subject  to  the  constraint  g  —  f  >  0  and 
such  that  g  —  f  <  e  —  10-3 .  That  is,  the  approximation  g  to  $  is  within  the 
given  tolerance  e  =  10-2  and  g  —  $  is  positive.  Figure  2  shows  g  (solid  curve) 


Figure  2.  Final  one-sided  approximation  after  knot  removal. 

and  the  error  g  —  $  (dashed  curve)  together  with  the  location  of  the  interior 
knots  left  by  the  constrained  knot  removal  method.  The  initial  knot  vector  has 
been  reduced  to 

r  =  (0,0, 0,0, 0.5, 1,1, 1,2, 2, 2, 2.5, 3, 3, 3, 4, 4, 4, 4). 

5.2  Convex  approximation. 


Figure  3.  The  initial  convex  approximation  to  the  turbine  data. 


The  dashed  curve  shown  in  Figure  3  is  a  cubic  spline  approximation  to  a  set  of 
data  points 

(*i,Vi)  =  (2«,yi),  for  t  =  1,  2,  ...,  50, 


where  y ,  is  measured  electricity  when  2 1  percent  of  maximum  power  is  forced 
upon  a  particular  water  turbine.  A  large  number  of  turbines  have  been  tested 
and  data  measured,  and  all  turbines  have  similar  curves  depending  on  the  turbine 
type,  the  size  of  the  turbine  and  various  other  factors.  These  curves  make  it 
possible  to  choose  the  right  turbines  given  the  water  resources  and  the  need 
for  electric  power.  Physical  considerations  indicate  that  such  curves  should  be 
convex,  and  with  a  tolerance  of  e  =  1.0  there  is  clearly  scope  for  knot  removal, 
so  it  is  natural  to  apply  knot  removal  with  convex  constraints. 

The  dashed  curve  <j>  shown  in  Figure  3  is  in  fact  the  Schoenberg  variation 
diminishing  approximation  to  the  data  on  the  knot  vector 

t  =  (ti)f4  =  (2, 2, 2, 2, 6, 8, ... ,  94, 96, 100, 100, 100, 100). 

The  interior  knots  are  indicated  at  the  top  of  Figure  3.  Our  problem  is  to  con¬ 
struct  a  spline  approximation  g  to  <p  subject  to  the  constraints 

g(ti)  =  y(<54)  =  g"(x)>0  for  all  x  6  [2, 100] . 

First,  we  computed  an  initial  spline  approximation  f  on  t  which  solves  (3)  with 

F 4,<  =  |  d\  =  a!  &  d50  =  a50  &  A dj  >  0,  for  3  <  j  <  50  j, 

J 

where  (a,-)  and  (dj)  are  the  B-spline  coefficients  of  <f>  and  /  on  t,  and  Ad,  is 
defined  by 

A  dj  =  <Tj-.\dj  —  +  <7,)dj_j  +  <7  jdj—2i 

where  <Tj  =  (<j+3  —  tj)/3.  The  spline  /  is  shown  in  Figure  3  (solid  curve). 
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Figure  4.  The  final  convex  approximation  after  knot  removal. 


Figure  5.  The  second  derivative  of  the  initial  (left)  and  final  convex 
approximations. 

Applying  constrained  knot  removal  to  /  with  the  same  constraints  and  with 
tolerance  e  =  1.0,  we  get  the  final  cubic  spline  approximation  g.  This  spline  is 
shown  in  Figure  4  (solid  curve)  together  with  the  error  function  g  —  /  (dashed 
curve).  The  three  remaining  interior  knots  are  indicated  at  the  top  of  the  figure. 

Figure  5  shows  the  second  derivative  of  /  and  g ,  and  we  observe  that  g  is 
smoother  than  / .  We  also  observe  that  the  computed  convex  approximation  to 
the  given  data  consists  almost  of  two  straight  lines. 


Figure  6.  Initial  approximation  to  the  turbine  data  with  the  first 
derivative  bounded  from  below  by  a  positive  spline. 


5.3  Curved  constraints 

This  example  illustrates  that  it  is  possible  to  introduce  curved  constraints  into 


the  knot  removal  method.  The  initial  data  are  the  values  of  the  cubic  spline  <t>  on 
the  knot  vector  t  =  (t,)j4  of  the  previous  example.  We  axe  also  given  a  quadratic 
spline  h  on  the  same  knot  vector  (the  dashed  curve  in  Figure  8).  Our  problem  is 
to  construct  a  spline  approximation  g  to  <f>  with  as  few  knots  as  possible,  subject 
to  the  constraints 


g(h)  =  4>(ti),  g{tn)  =  <t>(tm),  g'(x)>h(x)  for  all  x  6  [2, 100] . 


The  tolerance  is  1.0  as  in  the  previous  example. 

First,  we  compute  an  initial  spline  approximation  f  on  t  which  solves  (3) 

with 


F4>t  =  dx  =  ax  &  d50  =  a50  &  >  t ,  for  2  <  j  <  50 }, 


where  aj  and  (e^)  are  the  B-spline  coefficients  of  <f>  and  h  on  t,  and  cr,  = 
(tj+ 3  —  tj)/ 3.  The  result  is  shown  in  Figure  6,  with  <f>  dashed  and  /  as  a  solid 
curve.  We  note  that  /  is  monotone  since  we  have  chosen  a  positive  h . 


Figure  7.  The  final  approximation  with  bound  on  the  derivative  after 
knot  removal. 

Figure  7  shows  the  result  of  applying  constrained  knot  removal  to  /  with 
the  given  tolerance  e  =  1.0.  The  dashed  curve  is  the  error  function  g  —  f ,  and 
the  solid  curve  is  the  final  spline  approximation  g  to  <t> .  The  number  of  interior 
knots  was  reduced  from  46  to  9 ,  and  the  location  of  the  remaining  interior  knots 
is  marked  at  the  top  of  the  figure. 

In  Figure  8  the  constraining  function  h  (dashed  curve)  and  g'  (solid  curve) 
are  plotted.  We  note  that  the  peak  in  g'  is  induced  from  the  rapid  increase  in 
the  data  <f>  in  that  area. 
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Figure  8.  The  first  derivative  of  the  final  approximation  and  its  lower 
bound. 
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Abstract  Data  is  frequently  captured  in  very  long  sequences  at  equal  intervals  in  the 
independent  variable.  This  situation  occurs,  for  instance,  in  medical  diagnosis  where 
various  quantities  (eg  intestinal  pressure  and  acidity)  are  regularly  recorded  at  short 
time  intervals  (seconds)  over  periods  of  many  hours.  There  are  consequent  difficulties 
in  storing  and  analysing  the  resulting  very  large  amounts  of  data.  Another  difficulty 
in  analysing  the  data  (to  discover  peaks,  zeros,  etc)  is  that  the  data  is  often  noisy. 
These  difficulties  are  reduced  by  first  fitting  a  smooth  mathematical  function  to  the 
data  and  then  working  instead  with  that  function.  We  consider  here  an  approximate 
least-squares  fitting  method  using  polynomial  splines  with  uniformly  spaced  knots  at 
some  of  the  data  points.  Advantage  is  taken  of  the  structure  of  the  normal  matrix 
to  develop  a  technique  that  is  near  real-time  in  that  it  produces  a  “running  fit”  over 
successive  blocks  of  data.  The  degree  to  which  the  method  is  approximate  can  be 
controlled.  The  algorithm  is  fast  in  operation  and  the  fits  have  local  properties. 

Key  words:  B-splines,  data  approximation,  least  squares,  medical  data  analysis 


1.  Introduction 

Tijskens,  Janssens,  Vantrappen,  De  Bondt  and  Vandewalle  [11],  in  considering  the 
automatic  analysis  of  medical  data  gathered  from  (digitised)  multi-channel  chart 
recorders,  have  presented  a  method  for  fitting  extremely  long  sequences  of  uniformly 
spaced  data.  They  seek  a  fit  in  the  form  of  a  cubic  spline  with  prescribed  uniform 
knots,  because  this  function  is  sufficiently  versatile  to  represent  such  data  adequately. 
Moreover,  this  form  is  convenient  for  subsequent  (medical)  analysis  of  the  tracings. 
Data  reduction  is  important  because  it  is  necessary  to  avoid  storing  the  very  large 
amounts  of  data  involved.  This  implies  that  the  fitting  should  be  performed  essen¬ 
tially  in  “real  time”,  that  is  as  the  data  is  gathered.  To  this  end  they  use  a  “window” 
technique.  Data  in  a  window  spanning  an  odd  number  of  spline  intervals  is  fitted 


by  a  spline  in  its  B-spline  form  in  the  least-squares  sense,  using  the  singular  value 
decomposition  (SVD)  (5,  page  293  et  seq].  The  central  coefficient  of  the  fit  is  retained, 
the  window  advanced  one  interval  and  the  process  repeated.  The  window  is  chosen 
large  enough  so  that,  to  the  required  accuracy,  the  coefficients  obtained  are  identical 
to  those  that  would  have  been  calculated  in  a  single  fit  to  all  the  data. 

The  SVD  is  a  powerful  tool  that,  in  this  case,  takes  no  advantage  of  the  structure 
of  the  spline-fitting  problem.  Consequently,  the  computation  time  is  unnecessarily 
long.  Since  the  knots  of  the  spline  are  prescribed,  each  B-spline  coefficient  is  a  linear 
combination  of  the  data  values  in  the  window.  The  multipliers  or  weights  in  the 
linear  combination  can  be  pre-computed  using  an  approach  described  below.  Thus 
the  method  is  reduced  to  the  evaluation  of  a  single  inner  product  for  each  coefficient, 
with  the  same  set  of  multipliers  in  each  case.  This  paper  describes  this  alternative 
approach. 

2.  Solution  method 

We  shall  let  x  denote  the  independent  variable  and  y  the  dependent  variable  (in  this 
case  time  and  the  medical  response  indicated  on  the  chart  recorder,  respectively). 
Data  values  {yr}  are  assumed  to  be  provided  sequentially  at  corresponding  points 
{xr}  having  a  uniform  spacing  h.  This  data  is  to  be  approximated  in  near  real-time 
by  a  polynomial  spline  s(x)  of  order  m  (degree  m  -  1)  with  uniformly  spaced  knots. 
A  knot  will  be  placed  at  every  pth  data  point,  and  thus  the  knot  spacing  will  be  ph. 
It  is  algebraically  convenient  to  scale  the  independent  variable  so  that  ph  =  1.  If  this 
is  done  the  scaled  data  spacing  is  p~l  and  the  knots  are  at  the  integers,  p  is  then 
the  data  density,  that  is  the  number  of  data  abscissae  per  unit  of  the  independent 
variable  x.  Henceforth  we  shall  work  only  in  the  scaled  variables. 

The  spline  s  will  be  represented  as  a  linear  combination  of  B-splines, 

1 

where  iVm(x)  is  the  normalized  fl-spline  of  order  m  with  the  knots  0, 1, ....  m  [2]. 

A  (linear)  data  fitting  problem,  in  which  the  fitting  function  is  expressed  as 
a  linear  combination  of  basis  functions  (here  the  B-splines),  can  be  formulated  by 
writing  the  model  equation  for  each  data  point,  giving  an  overdetermined  system  of 
linear  equations: 

Ac=y.  (1) 

In  ( 1) ,  each  row  of  the  observation  matrix  A  contains  values  of  the  basis  functions 
at  a  data  point,  y  is  the  vector  of  corresponding  data  values  and  c  is  the  vector  of 
coefficients  to  be  determined.  The  normal-equations  approach  [6,  page  121  et  seg] 
to  obtaining  the  least-squares  solution  to  (1)  consists  in  forming  and  solving  the 
equations 


g 


r* 


where  H  =  ATA  and  r  =  ATy. 

The  system  of  equations  (2)  has  a  unique  solution  and,  equivalently,  H  is  invert¬ 
ible  if  and  only  if  A  has  full  column  rank. 

The  computation  falls  broadly  into  two  parts:  forming  H  and  inverting  it.  In  the 
present  application  it  is  particularly  simple  to  form  H  because,  with  uniform  knots, 
all  B-splines  (of  a  given  order)  are  translations  of  one  another.  So  the  element  in  row 
r  and  column  j  of  A  is  Nm(xr—j),  and  it  follows  from  the  compact  support  property 
of  B-splines  [2]  and  the  uniform  spacing  of  the  data  that  H  is  a  (2m  —  l)-diagonal 
Toeplitz  matrix  (that  is  the  elements  on  each  diagonal  are  constant).  A  row  of  H 
takes  the  form 


{...,  0,  hm  —  li  2i  ^li  ^0)  ^1»  •  ■  •  j  hm—  1 1  •••}, 


with 


^  =  X>">(*r)AUxr-». 


Because  of  the  uniform  knots  and  data,  H  is  well  conditioned  and  invertible  for 
moderate  values  of  m  provided  p  is  greater  than  unity.  So  the  usual  objection  that  the 
formation  and  solution  of  the  normal  equations  can  introduce  errors  that  are  greater 
in  magnitude  than  those  we  expect  with  a  stable  algorithm  is  of  no  significance  here. 

Consider,  as  an  example,  the  piecewise  linear  case,  m  =  2.  Here 


so  that 


and 


(  x,  x  6  [0,  1), 

N2(x)  =  i  2  -  x,  x  €  [1,  2], 
(  0,  otherwise, 


hl=J^N2(xr)N2(xT-l) 

4khh 

-  35C!  - 


ho  =  ^iV2(xr)N2(xr) 


-U^4rHr- 


=  35<4'j  +  2>- 

The  typical  row  of  H  for  B-splines  of  other  orders  can  be  obtained  similarly,  using 
the  basic  recurrence  relation  for  B-splines  [1]  and  formulae  for  the  sums  of  powers  of 
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Table  i 

Expressions  for  the  nonzero  elements  in  the  typical  row  of  the  normal  matrix  U . 


the  natural  numbers,  or  by  more  sophisticated  methods.  The  linear,  quadratic  and 
cubic  cases  are  given  in  Table  1,  where,  for  order  m,  each  element  is  multiplied  by 
(2m  —  l)!p2m-3  giving  even  polynomials  of  degree  2m  —  2. 

Various  properties  of  these  polynomials  are  readily  demonstrated.  In  particular, 
we  note  that  the  leading  coefficients  are  the  values  of  N2m  at  the  knots,  the  trailing 
coefficients  are  proportional  to  the  binomial  coefficients  of  order  2m  -  2,  and  the 
coefficient  of  p2"*-4  is  zero  (if  m  >  2).  The  first  observation  is  analogous  to  the 
integral  result  [10] 

r  '  Qdx  =  Mr+t(j  -  *),  (3) 

J  —  OO 

with  r  =  t.  In  (3),  MT  =  rNr  is  the  B-spline  with  integral  normalization 

f?00Mr(z)dx  =  1  [8]. 

The  third  observation  means  that  (for  m  >  2)  the  leading  term  dominates  by 
a  factor  0(p4).  This  factor  gives  an  indication  of  the  convergence  of  the  solution 
of  the  discrete  least-squares  spline  approximation  problem,  as  considered  here,  to 
that  of  the  continuous  uniform-knot  spline  approximation  problem  of  minimizing 
-*(*>>**• 

The  second  part  of  the  calculation,  obtaining  the  inverse  G  of  H,  can  also  be 
variously  accomplished.  Yamamoto  and  Dcebe  [12]  give  a  direct  formulation  for  the 
inversion  of  band  matrices  which  is  particularly  simple  for  tridiagonal  matrices.  In 
our  linear  case,  m  =  2,  this  gives  for  the  »,  j-th  element  of  G  (symmetric) 

(or*  -  /31)(an+1/^  -  aJ/?n+1)  .  ^  . 

9i'}~  (a -/?)(«»+»  -/?»+>)  ’  ~J' 

where  a  and  (3  are  the  roots  of 

Ajt2  +  hot  +  h\  =  0 


i 


and  n  is  the  dimension  of  H . 

This  is  equivalent,  for  sufficiently  large  n,  to  a  row 

{•••i  92,  9\,  9o,  9i>  92,  ■••} 


of  G  being  given  by 


9o  = 


gj-\  = -kgj  -  9j+ 1,  j  >  2,  fc  = /loMi  =  (4p2  +  2)/(p2  + 1). 

The  sequence  {<jj}  can  be  calculated  by  setting  9n+i  =  0,  gs  =  1  for  some  (no¬ 
tional)  N,  and  applying  the  recurrence  above  for  decreasing  values  of  the  suffix  until 
a  9N-M  is  calculated  with  a  sufficient  number  of  significant  figures.  The  sequence  is 
then  renumbered  and  normalized  so  that  9n~m  is  go  =  1  /v/(p2  +  2)/3. 

The  typical  row  of  G  for  this  order  of  spline  is  shown  to  6  decimal  places  for  p 
=  2,  4,  10  in  the  left-hand  part  of  Table  2.  For  p  =  10,  the  decay  factor  k  is  close  to 
its  lower  limit  of  2  4-  \/3  «  3.732. 

The  Yamamoto  and  Ikebe  algorithm  is  less  straightforward  for  m  >  2.  An  alter¬ 
native  approach  is  to  use  software  from  any  of  the  standard  linear  algebra  algorithms 
libraries  such  as  NAG  [4]  or  LINPACK  [3].  For  demonstration  purposes  we  used  the 
INV  function  from  PC-MATLAB  [7].  This  employs  an  LU  decomposition  of  H  and 
then  inverts  the  factors.  The  typical  (symmetric  half-)  rows  of  G  for  m  =  4  (cubic) 
with  p  =  2, 4, 10  are  shown  in  the  right-hand  part  of  Table  2.  The  decay  factor  is 
close  to  2  in  all  three  cases.  In  fact  we  can  see  from  Table  1  that  the  decay  tends  to 
2416/1191  «  2.0285  as  p  increases. 

Finally,  the  coefficients  of  the  fit  are  obtained  by  multiplying  an  appropriate 
(symmetric)  section  of  the  typical  row  of  G  into  the  corresponding  section  of  z.  The 
choice  of  this  section  is  equivalent  to  choosing  the  number  of  spline  intervals  in  the 
window  of  the  procedure  of  Tijskens  et  al  [11].  It  should  be  emphasised  that  once 
the  order  m  and  data  density  p  have  been  chosen,  the  calculation  of  the  typical  row 
of  G  is  only  required  once  and  cam  be  pre-computed.  This  gives  am  a  priori  estimate 
of  the  number  of  data  points  in  a  window  needed  to  achieve  a  given  accuracy  of  fit. 

A  refinement  of  the  method  is  to  compute  X  =  GAT  (the  pseudoinverse  [6,  paige 
36  seq\  of  A),  either  directly  or  as  the  product  of  G  amd  AT .  The  coefficients  of  the 
fit  are  then  obtained  as  the  scadar  product  of  an  appropriate  part  of  a  row  of  X  and 
the  corresponding  part  of  y.  Table  3  shows  part  of  the  (symmetric  half-)  rows  of  X 
for  m  =  4  amd  p  =  2, 4, 10  produced  using  PC-MATLAB.  The  elements  of  X  exhibit 
a  “piecewise”  decay,  and  it  is  again  straightforward  to  determine  the  length  of  the 
section  of  a  row  of  X  (equal  to  the  number  of  data  points  in  a  window)  required  to 
achieve  a  given  accuracy. 

In  the  above  we  have  implicitly  assumed  that  the  range  over  which  the  response 
is  measured  (and  fitted)  is  of  infinite  extent.  It  follows  that  the  matrices  A,  H,  etc 
ame  of  infinite  order.  Obviously,  the  data  is  gathered  on  a  finite  range.  Equally 


I 


P  =  2 

P  =  4 

p  =  10 

0.000001 

0.000000 

0.000000 

-0.000001 

-0.000001 

0.000000 

0.000002 

0.000002 

0.000001 

-0.000005 

-0.000003 

-0.000001 

0.000009 

0.000006 

0.000002 

-0.000017 

-0.000010 

-0.000004 

0.000032 

0.000019 

0.000008 

-0.000060 

-0.000036 

-0.000015 

0.000114 

0.000068 

0.000027 

-0.000215 

-0.000127 

-0.000051 

0.000407 

0.000237 

0.000095 

-0.000768 

-0.000443 

-0.000178 

0.001450 

0.000828 

0.000333 

-0.002737 

-0.001547 

-0.000622 

0.005168 

0.002891 

0.001162 

-0.009758 

-0.005403 

-0.002170 

0.018425 

0.010098 

0.004055 

-0.034791 

-0.018872 

-0.007575 

0.065690 

0.035270 

0.014151 

-0.124025 

-0.065913 

-0.026434 

0.234093 

0.123144 

0.049365 

-0.441263 

-0.229778 

-0.092074 

0.827044 

0.426393 

0.170791 

-1.511359 

-0.771993 

-0.309097 

2.445129 

1.240342 

0.496466 

-1.511359 

-0.771993 

-0.309097 

0.827044 

0.426393 

0.170791 

P  =  2 

p  =  4 

p  =  10 

-0.000000 

-0.000001 

-0.000001 

0.000000 

0.000004 

0.000003 

-0.000003 

-0.000018 

-0.000015 

0.000018 

0.000078 

0.000057 

-0.000105 

-0.000327 

-0.000217 

0.000612 

0.001363 

0.000824 

-0.003571 

-0.005672 

-0.003132 

0.020815 

0.023595 

0.011893 

-0.121320 

-0.098146 

-0.045163 

0.707106 

0.408248 

0.171498 

-0.121320 

-0.098146 

-0.045163 

0.020815 

0.023595 

0.011893 

-0.003571 

-0.005672 

-0.003132 

0.000612 

0.001363 

0.000824 

-0.000105 

-0.000327 

-0.000217 

0.000018 

0.000078 

0.000057 

-0.000003 

-0.000018 

-0.000015 

0.000000 

0.000004 

0.000003 

-0.000000 

-0.000001 

-0.000001 

Tabus  2 

The  left-hand  part  of  the  table  gives  the  values  of  the  elements  in  the  typical  row  ofG,  the  inverse  of 
the  normal  matrix  H ,  in  the  case  m  =  2  (spline  of  degree  1)  for  three  values  ofp,  the  data  density. 
The  right-hand  part  is  the  counterport  of  this  in  the  case  m  =  4  (cubic  spline),  showing  (symmetric) 
half-rows. 
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Tabus  3 

The  values  of  the  elements  in  the  typical  (symmetric)  half-row  of  X,  the  pseudoinverse  of  the 
observation  matrix  A,  in  the  casern  =  4  ( cubic  spline)  for  three  values  of  p,  the  data  density.  Note 
the  “ piecewise  decay”  in  the  magnitude  of  the  elements. 
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obviously,  the  observation  matrix  A  is  finite  and  not  all  its  rows  have  the  same  non¬ 
zero  elements,  because  the  first  few  and  last  few  rows  are  curtailed.  These  end  effects 
lead  to  a  matrix  G  whose  first  few  and  last  few  rows  are  not  shifted  copies  of  a 
typical  row.  However,  for  sufficiently  large  orders  of  matrix,  the  rows  converge  to  a 
central  section  of  the  matrix  whose  rows  do  have  this  property  to  working  precision. 
Furthermore,  to  that  precision  the  typical  row  is  the  typical  row  of  the  infinite  case. 
The  required  order  is  quite  modest;  the  examples  given  above  were  computed  with 
matrices  of  order  40. 


3.  Conclusion 

We  have  described  a  near  real-time  algorithm  for  fitting  arbitrarily  long  sequences  of 
uniformly-spaced  data.  The  method  uses  splines  with  uniform  knots  and  achieves  an 
approximation  to  the  least-squares  fit  to  all  the  data.  If  sufficient  data  values  “local 
to”  any  particular  B-spline  coefficient  in  the  approximant  s  are  employed  then,  to 
any  specified  precision,  s  can  be  regarded  as  the  least-squares  fit.  In  particular,  for 
practical  purposes,  the  accuracy  of  the  data  indicates  the  number  of  data  values  to 
be  employed.  For  instance,  if  the  data  density  p  is  10,  the  data  is  accurate  to  approx¬ 
imately  two  significant  figures,  and  a  cubic  spline  is  used,  each  B-spline  coefficient 
can  be  formed  as  a  linear  combination  of  about  150  of  the  data  values.  However, 
there  is  one  “replacement”  B-spline  coefficient  in  this  case  for  every  10  data  values. 
Hence  each  data  value  is  used  in  each  of  15  B-spline  coefficient  evaluations.  The  total 
number  of  floating- point  operations  is  therefore  only  15  times  the  number  of  data 
points.  Similar  statements  can  be  made  for  other  values  of  m  and  p.  The  multipli¬ 
ers  in  the  linear  combination  are  formed  by  a  straightforward  pre-computation  that 
depends  on  p  and  the  data  accuracy. 

The  case  p  equals  1  corresponds  to  placing  a  knot  at  every  data  point  and  hence 
to  interpolation.  Of  course,  this  is  a  much-studied  problem  dating  from  the  seminal 
work  of  Schoenberg  [8,  9].  In  one  sense,  the  ideas  here  are  a  straightforward  gener¬ 
alization  of  Schoenberg’s  work.  Considerable  work  needs  to  be  done,  particularly  on 
its  theoretical  aspects,  to  develop  this  generalization  fully. 
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AN  ALGORITHM  FOR  KNOT  LOCATION  IN  BIVARIATE  LEAST  SQUARES 
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Abstract  An  automatic  algorithm  for  the  determination  of  a  lattice  suitable  for  the 
construction  of  bivariate  least  squares  splines  is  studied.  Some  numerical  examples 
are  quoted. 

Key  words:  Bivariate  splines,  Knot  location,  Least  squares  approximation. 


1.  Introduction 

The  problem  of  constructing  a  surface  F(x,y ),  (x,  y)  6  D  C  722,  from  a  discrete 
number  of  points  (ij,  y<,  /,)  (i  =  1, . . . ,  N)  has  been  studied  by  many  authors.  In  the 
literature  there  appears  to  be  greater  concentration  on  the  development  of  algorithms 
for  data  interpolation  than  for  approximation  of  data  subject  to  errors,  where  we  are 
faced  with  additional  difficulties. 

When  spline  functions  are  employed,  it  is  well-known  that  cubic  splines  afford 
a  good  balance  between  efficiency  and  goodness  of  fit.  These  functions  are  gener¬ 
ally  written  in  terms  of  the  tensor  product  of  B-spline  basis  functions  {£/(x)}  with 
{Bm(y)}  defined  on  the  Cartesian  axes. 

In  the  case  of  cubic  spline  interpolation,  an  interpolant  to  data  on  any  rectangular 
grid  can  be  constructed.  If  the  grid  is  defined  by  the  lines  x  =  x*  («  =  1, . . . ,  JVX),  y  = 
Vi  O' *  If*.  N„),  then  the  points  x,-  and  yj  determine  the  knots  for  the  interpolating 
spline. 

When  the  data  values  contain  random  errors,  we  may  construct  an  approximation 
by  the  method  of  least  squares.  The  approximating  function  is  written  in  terms  of 
basis  functions,  fewer  in  number  than  the  data  points,  defined  on  a  knot  set  which  is 


different  from  the  original  data  point  set.  The  choice  of  knots  can  greatly  affect  how 
well  the  surface  fits  the  data. 

In  this  paper  an  algorithm  is  presented  for  the  automatic  placement  of  the  knots 
in  the  case  of  bivariate  least  squares  spline  approximation.  Before  describing  this 
algorithm,  we  briefly  review  in  the  next  section  how  to  construct  such  approximations. 


2.  Bivariate  least  squares  spline  approximation 

Consider  a  rectangle  R  =  {(x,  y)  €  R2  :  x  €  £a,  6],  y  €  [c,  <£]}  and  the  lattice  defined 
by  the  knots  {A,}£JJ  and  o>  where 

a  =  A0  <  Aj  <  . . .  <  Afc+i  =  b 

and 

c  =  no  <  ^  <  . . .  <  ng+i  =  d. 

A  bivariate  spline  of  order  k  on  R  with  interior  knots  {A,}^  and  is  a 

function  s(z,  y)  with  the  properties  that: 

i)  On  the  subrectangle  Rij  =  [A,_lt  A,]x(p,_t,/ij]  (»  =  1,...,A+1;  j  =  l,...,y+l), 
s(x,  y)  is  a  polynomial  of  total  order  2k  in  x  and  y. 
ii)  The  functions 

gp+q 

^^s(x,y)  {0<p,q<k-2) 

are  continuous  on  R. 

A  generally  well-conditioned  basis  for  s(x,  y)  is  given  by  the  tensor  product  of  kih 
order  univariate  B-splines  Af,(x)  with  knots  A„  (u  =  1, . . . ,  A)  and  Nj (y)  with  knots 

Pv  (v=  lf-.jf): 

k+k g+k 

s{x,y)  =  Y*lLvijMAx)Ni(y). 

•=i  ;=i 

The  least  squares  spline  fit  to  the  given  data  is  the  solution  of  the  problem 

(/*■  -£*'.jA/,(xr)lVi(yr) 

Equivalently,  we  may  solve,  in  the  least  squares  sense,  the  overdetermined  system  of 
equations 


Av  =  /, 


(1) 


where  v  =  expressed  as  a,  (h  +  k){g  +  fc)-ve ctor,  f  —  {fT)i  and  A 

is  the  N  x  (h  +  k)(g  +  k)  matrix  whose  rth  row  contains  the  basis  function  values 
Mi(xr)Nj{yr),  with 


N  >  (h  +  k)(g  +  k).  (2) 

A  solution  can  be  found  using  a  numerically  stable  method  which  employs  House¬ 
holder  transformations.  These  reduce  the  matrix  A  to  triangular  form  by  means  of 
orthogonal  transformations.  Another  method  is  based  on  Givens  rotations.  These 
are  particularly  useful  in  the  case  of  structured  matrices,  are  numerically  stable  and 
reduce  the  storage  requirements. 

The  major  difficulty  that  occurs  in  determining  the  least-squares  solution  of  (1)  is 
the  possible  rank  deficiency  of  the  matrix  A  which  depends  on  the  lattice  used.  In  the 
univariate  case,  A  has  full  rank  if  the  Schoenberg- Whitney  conditions  are  satisfied 
(Cox,  1986;  de  Boor,  1978),  while  in  the  bivariate  case  conditions  exist  only  for 
particular  configurations  of  the  points  and  knots.  Rank  deficient  matrices  occur  very 
frequently  and  lead  to  non-unique  solutions.  We  can  determine  a  particular  solution 
selecting  either  the  minimum  norm  solution  (which,  however,  is  not  invariant  under 
translation)  or  a  geometrically  invariant  solution  (Cox,  1986). 


3.  Knot  placement 

From  what  we  have  said  above,  a  need  clearly  arises  for  subdividing  the  rectangle  R 
into  a  lattice  that  affords  a  good  balance  between  the  closeness  of  the  fit  to  the  data 
and  the  smoothness  of  the  fit,  whilst  reducing  the  number  of  empty  panels. 

In  the  literature  we  find  results  for  data  on  a  mesh  and  scattered  data.  In 
the  former  case  an  algorithm  due  to  Dierckx  (1981)  can  be  used.  This  algorithm 
constructs  a  lattice  in  such  a  way  as  to  control  the  above-mentioned  balance. 

When  the  data  are  scattered  over  the  domain,  the  literature  suggests  methods 
which  are  based  on  sequences  of  “trials” .  More  precisely,  one  forms  an  initial  rectan¬ 
gular  lattice  and  constructs  the  relevant  spline;  if  the  fit  is  not  satisfactory  in  some 
regions,  the  lattice  is  refined  appropriately  (see  Cox  (1986)). 


4.  The  proposed  algorithm 

Our  objective  is  to  construct  an  algorithm  which  determines  a  suitable  lattice  for  any 
point  configuration.  In  order  that  the  behaviour  of  the  fitted  surface  is  sensible  and 
in  particular  sufficiently  smooth,  our  algorithm  is  required 

i)  to  provide  a  subdivision  which  achieves  near-uniformity  in  the  point  distribution 
between  the  mesh  lines, 
ii)  to  impose  a  limitation  on  the  knot  spacing. 

The  limitation  in  ii)  is  controlled  by  an  index  L,  as  described  below. 


We  first  determine  a  functional  relationship  between  L  and  the  number  of  data 
points  N.  To  this  end,  we  consider  the  real  variable  n,  which  coincides  with  N  at 
the  points  of  integer  abscissa,  and  require  that  the  following  conditions  are  satisfied: 

lim  L  =  oo, 

n— o 

lim  L  =  constant, 

n— *  oo 

L  >  0  V  n  e  K+, 

L'  <  0  V  n  €  U+. 

As  a  first  approximation,  we  write 

L  =  (3) 

where  /?  and  7  are  positive  constants  which  depend  on  the  measure  n(R)  of  the 
domain,  and  a  6  ft+.  The  relationship  (3)  must  also  satisfy  the  following  two 
conditions: 

1)  The  decrement  in  L  must  not  in  magnitude  be  much  larger  than  the  relevant 

increment  in  N. 

2)  When  the  given  points  are  dense  in  the  domain,  L  must  be  nearly  insensitive  to 

variations  in  N. 

If  we  consider  a  value  n,  am  increment  An  in  n  and  the  corresponding  values 
Z,(n)  and  £(n  +  An),  we  have 

A L  =  L(n  +  An)  -  L(n)  =  £/(n)An  +  Rj{ An), 

where  A](An)  is  the  remainder  from  the  Taylor  formula.  From  this,  for  An  suffi¬ 
ciently  small, 

AL  =  L'(n)An  =  -^An. 

In  order  to  satisfy  the  first  of  the  above  conditions,  it  is  necessary  that  0  <  a  <  1; 
whereas,  for  the  second,  it  is  convenient  to  tadce  a  >  1.  As  a  consequence,  a  suitable 
value  is  a  =  1,  giving 

L  =  0^  +  r  (4) 

This  conjecture  is  corroborated  by  practice.  Therefore,  when  forming  the  lattice, 
we  shall  consider  (4)  and  the  first  assumption  i).  In  order  to  satisfy  the  latter,  we 
proceed  in  the  following  way. 

Let  us  think  of  the  points  as  point  masses  and  determine  their  barycentre.  If 
the  points  are  regularly  distributed,  the  barycentre  is  the  centre  of  symmetry  of  the 
system  and  its  coordinates  subdivide  the  domain  into  four  equal  and  homogeneous 
parts.  Otherwise,  the  barycentre  lies  in  the  zone  where  the  density  is  the  greatest; 


in  this  case,  the  subdivision  makes  the  four  zones  obtained  more  homogeneous  than 
any  other  subdivision. 

The  algorithm  works  in  the  following  way.  For  a  given  set  In  of  points  P,  = 
P(x,, y,)  (i  =  we  determine  the  point  €  In  which  has  the  minimum 

Euclidean  distance  from  the  barycentre  B.  We  then  find  a  circular  neighbourhood 
[/(P<0))  about  with  radius  r  depending  on  the  mean  density  of  the  data  in  the 
domain  R: 

N  " 

The  factor  k  depends  on  the  point  configuration.  If  we  have  some  information  on  the 
lattice,  the  user  can  assign  an  initial  value  k*.  Otherwise,  the  following  rule  can  be 
used. 

Suppose  the  N  data  points  are  placed  on  a  rectangular  grid  as  follows: 
a  =  X\  <  xj  <  . . .  <  xst  —  6, 

c  =  yi  <  jft  <  •  •  •  <  yjv,  =  d, 


and  define 


=  *i+i  -  Xi  =  (6  -  a )/{Nx  -  1), 


*2  =  yi+i  ~Vi  =  (d~  c)/(Ny  -  1). 


In  order  that  the  neighbourhoods  cover  the  domain,  it  is  sufficient  to  choose  the 
radius  such  that 

r  >  y/h\  +  h\. 


Hence, 

_  N  I  {b—  a)2  .  (d-c)2 

*-p(fi)V(iVr-l)i  +  (iVv-l)2- 


(5) 


Thus,  we  shall  thus  choose  an  initial  value  k*  for  k  satisfying  (5). 

Once  we  have  determined  U(P ^),  we  calculate  the  barycentre  B0  of  the  points 
Pi  €  In  H  U(P^°))  and  call  (xg0,  yg^)  the  coordinates  of  B0.  We  proceed  by  con¬ 
sidering  the  set  of  points  Pj  €  (In  -  U(P ^))  =  Ist ;  the  algorithm  determines  the 
neighbourhood  (/(P^),  with  radius  r,  of  the  point  F 6  /at,  which  has  the  mini¬ 
mum  distance  from  the  barycentre  B.  Then  we  calculate,  as  above,  the  barycentre 
B\  of  the  points  Pi  6  In  D  U(P ^).  This  procedure  is  repeated  until  there  are  no 
more  points  Pi  remaining.  If,  for  some  F*'\  we  have  U(F *W)  =  0,  this  point  is  not 
taken  into  consideration,  and  we  proceed  to  determine  the  next  point  closest  to  B. 

When  the  procedure  ends,  we  see  if  (2)  is  satisified;  if  not,  we  repeat  the  procedure 
iteratively  by  incrementing  the  value  of  k  until  (2)  is  satisfied. 


The  set  ABj  of  the  coordinates  (xgv  Vb,  )  of  the  barycentres  B,  (j  =  0 . m) 

can  be  considered  as  a  potential  set  of  knots  on  the  relevant  coordinate  axes,  defining 
a  first  lattice.  At  this  point,  the  algorithm  controls  the  knot  spacing  by  considering 
the  sequences  and  {yg0, . . . , ygm }  of  coordinates.  For  any  couple 

Bj,  Bj+i  consider  the  inequalities: 

ksJ+,  -  xB,l  >  l,  (6) 

|S/BJ+,  -  VB,\  >  L.  (7) 

If  (6)  and  (7)  are  satisfied,  the  knots  are  accepted.  Otherwise,  if  one  of  the  above 
relationships  is  not  verified  for  at  least  one  of  the  couples  Bj  and  Bj+ j,  we  proceed 
as  follows. 

Suppose  (6)  is  not  verified.  We  consider  the  knots 
XB,„V  xBj'  xB]Jt ,  and 

and  define  xj,  xj+i  and  xj+2  to  be  the  midpoints  of  the  intervals  [xg,,  Xg(+1]  (/  = 
j  -  1, . . . ,  j  +  2).  If  NXj  and  Nrj+I  are,  respectively,  the  number  of  points  (each  with 
its  multiplicity)  belonging  to  the  intervals  [x;, ij+i]  and  [xj+i,x;+2L  we  determine 
in  the  interval  [xg^,xgj+1]  the  point  x*  for  which  the  equality 

x  Nx,  =  (IBJ+1  -  XB,  -  x  )NIj+1 

holds.  The  sequence  of  knots  on  the  x-axis  is  then  modified  to 

*B0.*--»*B,_,.  **»  xBj+2,  -,XBm. 


5.  Numerical  results 

In  this  section  we  use  an  example  found  in  the  literature,  namely  the  test  function 
F(x,  y)  =  exp(-x2  -  y2).  The  function  is  assumed  specified  by  a  sample  of  values 

f,  =  Fj  +  G  U[- 0.5  x  10'3, 0.5  x  103], 

where  U  is  the  uniform  distribution  and  the  approximating  spline  is  that  of  minimum 
norm. 

The  error  indices  we  have  considered  are 

L.S.E.  =  yf£ej/N,  A.E.  =  |«*|/Ar  and  MAX  =  ^max^  |e,|. 


where  e<  =  F(xi,yi)  -  s(x;,yj). 
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Example  1  Example  2 


R  =  [0, 1]  x  [0, 1] 

R  =  [-1,1.5]  x  [- 

N  =  1000 

N  =  100 

L.S.E.  =  .1179E-03 

L.S.E.  =  .1091E  -  02 

A.E  =  .9284E  -  04 

A.E.  =  .8577 E  -  03 

MAX  =  .4642E  -  03 

MAX  =  .3078E  -  02 

6.  Concluding  remarks 

• 

For  small  data  sets,  the  algorithm  presented  provides  good  results  both  for  data  on  • 

a  mesh  and  for  scattered  data.  For  large  data  sets,  our  results  compare  favourably 

with  those  provided  by  methods  discussed  in  the  literature.  The  comparison  holds 

also  for  computation  time.  Moreover,  memory  requirements  are  not  excessive,  so  the 

algorithm  can  be  used  on  a  personal  computer  and  can  be  modified  for  interactive 

use.  0 
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A  KNOT  PLACEMENT  STRATEGY  FOR  LEAST  SQUARES  SPLINE  FITTING 
BASED  ON  THE  USE  OF  LOCAL  POLYNOMIAL  APPROXIMATIONS 


M.  G.  COX,  P.  M.  HARRIS  and  Helen  M.  JONES 
National  Physical  Laboratory 


Abstract  We  are  concerned  with  the  use  of  univariate  spline  functions  in  fitting 
noisy  data  in  the  least-squares  sense.  When  defining  the  linear  space  of  approximating 
functions  from  which  our  fit  is  to  be  taken,  there  is  freedom  both  in  the  number  and 
locations  of  the  interior  knots.  The  distribution  of  these  knots  can  have  a  profound 
influence  on  how  well  the  spline  fits  the  data. 

We  describe  an  algorithm  for  determining  an  initial  estimate  of  the  number  of 
knots,  and  their  distribution,  from  which  an  initial  fit  to  the  data  is  obtained.  The 
algorithm  exploits  in  a  natural  way  the  piecewise  polynomial  nature  of  the  approx¬ 
imating  function  by  isolating  subsets  of  the  data  which  are  adequately  described  by 
simple  polynomials.  Once  an  initial  set  of  knots  is  known,  adaptive  knot  placement 
strategies  may  be  used  to  improve  this  set. 

Key  words:  Knot  placement,  Least  squares,  Local  polynomial,  Polynomial  spline. 


1.  Introduction 

Polynomial  spline  functions  are  widely  used  for  fitting  “noisy”  data;  that  is,  data  in 
which  there  are  random  errors  in  the  dependent  variable  values.  In  practice,  such  data 
can  arise  when  these  values  are  measurements  of  the  response  of  a  physical  system  to 
known  discrete  inputs  (the  independent  variable  values).  Using  the  method  of  least 
squares,  a  polynomial  spline  approximation  to  the  underlying  function  represented 
by  the  data  can  be  constructed. 

When  defining  the  linear  space  of  splines  from  which  the  fit  is  to  be  taken,  there 
is  considerable  freedom  of  choice  in  both  the  order  n  of  the  spline  and  in  the  number 
and  locations  of  the  interior  knots  Ay,  j  =  1, . . . ,  N.  We  are  interested  in  constructing 
strategies  and  algorithms  for  automatically  choosing  A  =  (Aj,...,Ajv)  so  that  the 
spline  fit  of  given  order  with  these  knots  is,  in  some  sense,  an  “acceptable”  fit  to  the 
data. 


In  Cox,  Harris  and  Jones  (1987)  we  discuss  various  approaches  to  solving  the 
problem  of  automatic  knot  placement.  Algorithms  for  deriving  an  initial  trial  set 
of  knots  and  adaptive  knot  placement  strategies  are  both  considered.  In  particular, 
we  present  there  a  new  version  of  an  algorithm  given  in  de  Boor  (1978)  which  we 
summarise  below. 

Let  the  data  consist  of  the  points  (x„  /,),  i  =  1, . . . ,  m,  where  x,  €  [xm,m  xmax ] 
for  all  i.  Suppose  that  we  require  a  spline  fit  of  order  n  (degree  at  most  n  —  1)  to 
the  data  and  we  wish  to  position  N  interior  knots  in  (xmm,  xmax).  We  first  define 
exterior  knots  A j  =  xm,„  for  j  <  1,  A j  =  xmax  for  j  >  N  and  then,  according  to  this 
strategy,  we  try  to  choose  Aj,  j  =  1, . . . ,  N,  such  that 

/A‘+1  |/(n,(x)|1/n  dx  =  -j—  r~  |/(")(x)|1/»dx,  (1) 

-T  1  Jxmtn 

for  jfc  =  0, . . . ,  N.  We  assume  that  /,  the  (unknown)  underlying  function,  belongs  to 
the  continuity  class  Cn(xmin,  xmarj.  The  criterion  (1)  gives  an  interior  knot  distribu¬ 
tion  which  is  asymptotically  equivalent,  as  N  — *  oo  with  maxjt=0,...,/v  |A*+1  —  A* |  — ►  0, 
to  that  obtained  from  solving 

mt  </.*>}  •  (2) 

In  (2),  udist[AjkjAjk+1](/,5^)”  represents  the  “distance”,  measured  in  the  infinity  norm 
over  [A*,  A*+1],  between  /  and  5^,  the  linear  space  of  splines  of  order  n  with  interior 
knot  vector  A.  Criterion  (1)  is  derived  from  (2)  by  replacing  this  distance  function 
by  a  suitable  bound. 

Our  version  of  the  algorithm  differs  from  de  Boor’s  in  that  we  approximate 
by  a  piecewise  linear  function  rather  than  by  a  piecewise  constant  function.  As  a 
result  we  hope  to  achieve  a  better,  and  certainly  smoother,  approximation.  The 
strategy  is  adaptive  since  the  approximation  to  /(")  is  obtained  from  a  spline  fit 
to  the  data  based  on  a  current  set  of  interior  knots.  Thus,  one  application  of  the 
criterion  results  in  this  current  set  being  updated,  and  the  new  knot  vector  forms  the 
current  knots  at  the  next  application. 

In  order  to  use  (1)  adaptively,  we  need 

(a)  an  initial  interior  knot  vector  A, 

(b)  a  means  of  approximating  /*n*  using  the  current  set  of  knots,  and 

(c)  a  means  of  terminating  the  algorithm. 

Details  of  how  the  approximation  to  /(")  is  constructed  is  given  in  Cox  et  al  (1987). 
Observe  that  (a)  will  need  to  be  found  independently  of  a  current  knot  set  since,  to 
begin  with,  no  such  set  will  exist.  Moreover,  the  effectiveness  of  this  algorithm  will 
depend  on  the  quality  of  the  initial  approximation. 

In  this  paper  we  present  an  algorithm  for  determining  initial  values  for  N  and 
A  j,  j  =  1,  ...,1V.  The  idea  is  to  try  to  deduce  information  about  the  underlying 
function  /  from  simple  polynomial  fits  to  subsets  of  the  complete  data  set.  These 
“local”  fits  give  us  “local”  information  about  /  independently  of  a  set  of  interior 


knots.  We  then  use  this  information  to  construct  A.  We  assume  throughout  that 
there  is  enough  data  to  ensure  that  proper  subsets  which  adequately  describe  /  exist. 
Once  an  initial  set  of  knots  is  known,  the  adaptive  knot  placement  strategy  described 
above  may  be  used  to  improve  this  set. 

In  Section  2  we  present  an  algorithm  which  associates  a  local  data  set  and  a  cor¬ 
responding  local  polynomial  fit  with  each  point  “sufficiently  interior”  to  the  complete 
data  set.  We  show  in  Section  3  how  this  local  information  may  be  used  in  a  strategy 
for  knot  placement.  Results  of  using  the  strategy  are  also  presented  in  this  section. 
Finally,  in  Section  4  we  summarise  the  work. 


2.  Deriving  Local  Information  About  / 

Given  m  data  points  (x„ /)),»'  =  1, ...,m,  with  xj  <  ...  <  xm,  representing  an 
unknown  underlying  function  /,  we  show  how  we  may  associate  with  each  data  point 
sufficiently  interior  (see  later)  a  local  data  set  and  a  local  polynomial  fit  of  order  n. 

Consider  the  ith  point  (x,,  /,).  We  first  construct  a  subset  of  contiguous  points 
taken  from  the  complete  data  set,  with  indices  centred  on  i,  and  consisting  of  n  +  1 
points  for  n  even  or  n  points  for  n  odd.  We  regard  this  subset  as  local  to  the  ith  point. 
It  is  assumed  that  these  points  can  be  “adequately”  fitted  by  a  simple  polynomial  of 
order  n.  This  will  certainly  be  true  in  the  case  when  n  is  odd  since  we  can  interpolate 
the  local  data  by  such  a  polynomial.  It  is  now  also  evident  what  we  mean  by  a  point 
being  sufficiently  interior  -  it  is  a  point  for  which  this  initial  local  data  set  can  be 
constructed. 

Our  next  step  is  to  increase  the  size  of  the  subset  by  adding  the  neighbouring 
point  at  each  end  if  this  can  be  done.  A  polynomial  fit  of  order  n  to  the  new 
local  data  is  computed  and  Powell's  trend  test  (Powell,  1970)  is  used  to  assess  the 
“acceptability”  of  the  fit. 

The  largest  subset  so  formed  for  which  the  polynomial  fit  to  this  data  satisfies 
the  test  is  taken  to  be  the  local  data  set  associated  with  the  ith  point.  The  procedure 
is  then  repeated  for  all  sufficiently  interior  points. 

In  the  use  of  the  algorithm  as  described,  there  can  be  a  problem,  particularly 
for  large  data  sets,  with  regard  to  the  time  taken  for  its  execution.  This  is  because 
we  may  need  to  compute  a  large  number  of  polynomial  fits  (and  evaluate  these  at 
every  point  of  the  corresponding  data  set  for  use  in  the  trend  test)  for  each  of  a  large 
number  of  points. 

To  reduce  the  amount  of  work  we  use  the  size  of  the  final  local  data  set  associated 
with  the  (i  —  1)‘A  data  point  as  a  starting  value  for  the  size  of  the  local  data  set  for 
the  ith  point.  This  set  is  then  enlarged  or  reduced  depending  on  whether  it  can 
adequately  be  fitted  by  a  polynomial  in  the  sense  described.  It  is  not  clear  whether 
we  can  conclude  that  if  the  fit  to  this  set  of  data  satisfies  the  trend  test  then  the  fits 
to  all  smaller  subsets  must  also  satisfy  the  test.  But  equally,  if  we  always  start  with 
the  smallest  subset  and  look  for  the  first  local  data  set  to  fail  the  trend  test,  it  is  not 
clear  that  all  larger  sets  must  also  fail  the  test.  Thus,  we  believe  that  reducing  the 


work  in  this  way  is  not  detrimental  to  the  results  produced  by  the  algorithm. 

The  execution  time  is  also  affected  by  the  manner  in  which  the  polynomial  fits  are 
computed.  Traditionally,  the  methods  used  to  generate  a  least-squares  polynomial 
fit  to  data  are  those  of  Forsythe  (Forsythe,  1957)  and  Forsythe-Clenshaw  (Clenshaw. 
1960).  We  use  Forsythe’s  method  in  preference  to  that  of  Forsythe-Clenshaw  because 
for  our  purposes  it  is  appreciably  faster. 
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FlG.  1.  A  subset  of  the  data  obtained  from  measurements  of  the  response  of  an  oscilloscope  to  a 
step  input  function  and  local  polynomial  fits  obtained  using  a  trend  test. 

Figure  1  shows  the  local  polynomial  fits  as  produced  by  the  algorithm  for  a 
set  of  data  generated  by  measuring  the  response  of  an  oscilloscope  to  a  step  input 
function.  The  complete  data  set  consists  of  512  points  with  equispaced  abscissa 
values  but  we  show  here  only  a  subset  of  the  data  in  order  to  illustrate  more  clearly 
the  local  polynomials.  The  first  diagram  shows  this  subset  and  the  second  the  local 
polynomial  fits  of  order  4  associated  with  these  and  neighbouring  points.  We  notice 
that  the  aggregate  of  the  simple  polynomial  fits  appears  to  be  adequately  describing 
the  underlying  function.  In  the  following  section  we  use  the  complete  data  set  in  our 
knot  placement  strategy. 


3.  An  Algorithm  for  Knot  Placement 


In  this  section  we  show  how  we  may  use  the  information  generated  by  the  algorithm 
presented  in  Section  2  to  determine  an  initial  trial  set  of  knots.  These  knots  are  used 
in  a  least  squares  spline  fit  to  the  data,  and  may  be  subsequently  updated  according 
to  (1)  to  give  improved  fits. 

Suppose  we  compute  the  local  data  sets  associated  with  all  sufficiently  interior 
data  points  using  polynomial  fits  of  order  n.  Let  the  first  and  last  data  point  in  each 
local  dataset  mark  the  lower  extent  and  upper  extent ,  respectively,  of  that  set.  Then, 
we  position  a  knot  at  any  point  which  is  the  lower  (or  upper)  extent  of  more  than 
two  local  data  sets  associated  with  consecutive  data  points.  In  this  way  we  derive 
not  only  an  estimate  for  N  but  also  for  the  initial  knot  positions  Aj,  j  =  1, . . . ,  N. 

If  the  local  data  set  for  the  ith  point  is  determined  independently  of  those  for  the 
other  points,  then  this  strategy  clearly  gives  a  vector  A  whose  elements  do  not  depend 
on  the  order  in  which  the  points  are  processed.  However,  we  observed  in  Section  2 
that  for  reasons  of  efficiency,  we  use  the  size  of  the  final  local  data  set  for  the  (i  —  I)** 
point  as  a  starting  value  for  the  size  of  the  local  data  set  for  the  ith  point.  Thus,  the 
points  are  necessarily  processed  from  left  to  right  and  there  is  a  dependence  between 
the  local  data  sets  for  consecutive  data  points.  A  better  criterion  is  to  locate  a  knot 
at  an  extent  of  a  local  data  set  which  is  greater  than  or  equal  to  the  extents  of  more 
than  two  subsequent  and  consecutive  local  data  sets.  Similarly,  if  the  points  were 
processed  from  right  to  left,  we  would  use  a  “less  than  or  equal  to”  test  here. 

Figure  2(a)  shows  the  result  of  applying  the  knot  placement  strategy  to  the 
oscilloscope  data,  part  of  which  is  illustrated  in  Figure  1.  The  data  points  (small 
crosses)  are  shown  together  with  the  knot  positions  (vertical  bars)  and  the  fourth 
order  spline  fit  to  the  data  with  these  knots.  There  are  70  knots  and  the  root  mean 
square  residual  in  the  fit  is  0.0543.  We  notice  that  in  some  regions  there  are  clusters 
of  knots  and  these  can  cause  overfitting  of  the  data.  We  say  that  overfitting  occurs 
when  the  spline  fit  passes  so  close  to  the  data  that  it  begins  to  model  the  noise. 

To  remove  the  clusters  we  use  a  simple  strategy  which  requires  that  each  knot 
interval  contains  at  least  n  (=  4,  in  this  example)  data  points.  If  there  are  fewer 
than  n  points  in  an  interval,  the  right-hand  knot  is  removed  (excepting  the  last  knot 
interval  where  it  is  the  left-hand  knot  which  is  removed).  As  a  result,  we  are  left  with 
44  knots  and  a  spline  fit  as  shown  in  Figure  2(b).  The  root  mean  square  residual  in 
this  fit  is  0-0605.  The  fit  looks  much  more  acceptable  than  that  of  Figure  2(a)  because 
the  oscillations  in  the  spline  that  were  previously  present  have  been  eliminated  or 
reduced. 

Finally,  if  we  use  these  44  knots  to  initialize  the  adaptive  knot  placement  algo¬ 
rithm  (1),  and  continue  to  remove  clusters  of  knots  as  they  are  generated,  we  produce 
after  4  iterations  the  knots  and  associated  spline  fit  shown  in  Figure  2(c).  There  are 
42  knots  here  and  the  root  mean  square  residual  in  the  fit  is  0.0618.  Indeed,  the  use 
of  (1)  has  produced  little  improvement  over  the  set  of  44  knots.  We  note,  however, 
that  the  distribution  of  the  reduced  set  of  knots  is  markedly  different. 

Figure  3  shows  the  results  at  the  same  stages  for  another  practical  set  of  data. 
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This  arises  from  very  accurate  measurements  of  the  response  of  a  photodiode.  The 
data  set  consists  of  1024  points  with  equispaced  x, -values.  For  clarity,  we  have 
plotted  only  every  fifth  point  in  the  diagrams  of  Figure  3.  A  set  of  79  knots  is 
initially  generated  and  this  is  reduced  to  55  when  clusters  are  removed  as  described 
above.  The  corresponding  fits  are  illustrated  in  Figures  3(a)  and  (b)  and  the  root 
mean  square  residuals  are  0.00280  and  0.00295,  respectively.  In  this  example,  the 
use  of  (1)  gives  a  new  set  of  knots  which  results  in  a  very  substantial  improvement 
in  the  root  mean  square  residual.  Figure  3(c)  shows  the  knots  and  corresponding 
spline  fit  after  4  iterations  of  adaptive  knot  placement.  There  are  34  knots  here,  and 
the  root  mean  square  residual  in  the  fit  is  2.73  x  10-7.  Adaptive  knot  placement  has 
reduced  the  root  mean  square  residual  by  a  factor  of  10,000  without  overfitting.  We 
observe  that  much  of  this  improvement  results  from  the  strategy  repositioning  the 
knots  in  such  a  way  as  to  remove  the  severe  oscillation  that  previously  existed  in  the 
“shoulder”  of  the  curve. 


4.  Summary 

In  this  paper  we  have  presented  a  strategy  for  automatically  deriving  a  knot  vector 
A  to  be  used  in  constructing  a  least  squares  spline  fit  of  given  order  to  noisy  data. 
We  determine  subsets  of  the  complete  data  set  which  are  adequately  fitted  by  sim¬ 
ple  polynomials.  These  polynomials  provide  local  information  about  the  underlying 
function  which  the  data  represents.  By  considering  how  these  subsets,  or  local  data 
sets,  interact  we  generate  initial  estimates  for  N  and  A j,  j  —  1, . . . ,  N.  With  these 
initial  values  we  may  then  use  an  adaptive  knot  placement  algorithm  to  produce 
improved  knot  distributions.  The  particular  adaptive  strategy  we  use  here  relocates 
knots  according  to  the  criterion  (1)  and  removes  knots  to  prevent  clustering.  The 
examples  we  have  given  illustrate  some  of  the  features  of  our  approach.  For  further 
details  and  examples,  see  Cox,  Harris  and  Jones  (1988). 
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AN  ALGORITHM  FOR  NONLINEAR  SPLINES  WITH  NON-NEGATIVITY 
CONSTRAINTS 
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Abstract  The  problem  of  minimizing  the  exact  form  of  the  strain  energy  of  a 
“spline”  subject  to  non-negativity  constraints  is  treated.  Several  necessary  opti¬ 
mality  conditions  in  terms  of  systems  of  differential  equations  are  derived.  Due  to 
the  constraints,  additional  boundary  conditions  have  to  be  introduced  at  points 
which  are  not  known  in  advance.  An  algorithms  is  outlined,  which  we  call  a  local 
algorithm,  which  is  based  on  only  two  grid  points. 

Key  words:  Curvature  functional,  Elastica  with  constraints,  Global  spline  algo¬ 
rithm,  Local  spline  problem,  Nonlinear  splines,  Non-negativity  constraints,  Ob¬ 
stacle  problem,  Splines  with  constraints. 

1.  Introduction 

In  many  applications,  splines  should  not  only  pass  through  given  data  points 
(tj,Xj),  j  =  1,2, ...,n  but  should  also  satisfy  some  additional  conditions  which 
are  prescribed  for  the  given  problem.  For  example,  if  one  wants  to  model  a 
density  function  [cf.  Feller  (1971,  p.3)]  the  model  is  useless  unless  it  can  produce 
non-negative  values.  A  more  general  type  of  restriction  is  one  which  requires  a  fit 
to  be  “visually  pleasing”  [cf.  Carlson  (1987)]  and  the  meaning  of  this  may  vary 
from  problem  to  problem. 

One  approach  to  such  a  problem  is  motivated  by  physical  arguments.  The 
curve  one  is  interested  in  is  regarded  as  a  flexible  ruler  or  beam  (draughtsman’s 
spline)  (cf.  Malcolm  (1977),  de  Boor  (1978,  p.67))  and  thus,  the  strain  energy 
has  to  be  minimized,  of  course  subject  to  the  given  restrictions. 

The  second  approach  is  motivated  more  by  practical  arguments.  In  this  ap¬ 
proach  one  constructs  a  spline  directly,  such  that  it  fits  all  the  given  requirements. 
This  is  ordinarily  done  by  choosing  additionally  introduced  parameters  suitably. 


For  splines  with  no  restrictions  apaxt  from  the  interpol&tory  conditions,  the 
first  approach  was  followed  for  example  by  Golomb  and  Jerome  (1982).  For  non¬ 
negativity  constraints  in  connection  with  the  simplified  strain  energy  f*  x(t)2dt 
see  Opfer  and  Oberle  (1988).  For  results  in  the  case  of  minimizing  f^{x^k\t)}2dt 
restricted  to  x^m^  >  0  see  Dontchev  and  Kalchev  (1988).  Recent  contributions 
to  the  second  approach  are  given  by  Rentrop  and  Wever  (1988)  and  Schaback 
(1989).  The  case  where  the  functional  to  be  minimized  and  the  constraints  are 
all  of  general  form  is  treated  by  Opfer  (1989). 

In  this  paper  we  shall  follow  the  concept  of  minimizing  the  strain  energy, 
which  is  expressed  by  the  integral  over  the  square  of  the  curvature  of  the  cor¬ 
responding  curve,  subject  to  non-negativity  constraints.  Thus  the  curves  to  be 
investigated  cannot  penetrate  the  x-axis,  and  this  type  of  problem  may  be  re¬ 
garded  as  an  obstacle  problem. 

For  brevity  and  simplicity  our  problem  will  be  posed  only  for  functions  rather 
than  curves. 

2.  The  Problem 

Given  data  ( tj,Xj ),  j  =  1,2, . . .  ,n  >  2,  with  a  =  t\  <  t2  <•■•<  tn  —  b  and  n 
fixed,  we  require  a  function  x  €  W2  with  x(t})  =  x},  j  =  1,2, ...  ,n,  which,  if 
parametrized  with  respect  to  arc  length  s,  minimizes 

/(x)  =  /  *2(s)ds  (1) 

Jo 

subject  to 

x(s)  >  0,  (2) 

where  k  is  the  curvature  of  x  and  t  the  length  of  the  curve  described  by  x.  It 
should  be  pointed  out  that  in  this  setting  neither  the  length  l  of  the  resulting 
curve  x  nor  the  interpolation  conditions  (which  have  the  form  x(s})  =  ij)  are 
known  in  advance,  since  the  Eire  lengths  Sj,  which  are  the  lengths  from  the  first 
point  to  the  point  numbered  j  measured  on  x,  axe  not  known  beforehand. 

It  is  mentioned  in  VEurious  papers  [e.g.  Golomb  and  Jerome  (1982)]  that  the 
corresponding  Euler  equation  for  the  unrestricted  case  can  be  put  (by  suitable 
tremsformation)  in  the  form  of  the  pendulum  equation,  with  the  consequence  that 
the  minimum  of  the  above  mentioned  functions^  (if  it  exists  at  eJI)  is  a  piecewise 
elliptic  integral.  However,  very  few  authors  use  this  result.  One  exception  is 
K.D.  Reinsch  (1981)  who  tsdees  advantage  of  the  existing  procedures  for  elliptic 
functions  presented  by  Bulirsch  (1965). 

Most  of  the  numerical  work  in  the  unrestricted  case  weis  discussed  by  MeJ- 
colm  (1977).  In  particular,  results  by  Glass  (1966),  Larkin  (1966),  Lee  and 


Forsythe  (1973),  Mehlum  (1969),  Woodford  (1969)  Me  mentioned.  We  are  not 
aware  of  any  more  recent  published  results. 

The  above  setting  in  terms  of  the  arc  length  s  has  the  advantage  of  formal 
simplicity;  the  disadvantage  already  mentioned  is  that  the  upper  bound  for  the 
integral  in  (1)  and  the  interpolation  conditions  contain  a  quantity  unknown  a 
priori.  In  this  paper  we  have  decided  to  use  rectangular  coordinates  and  restrict 
our  attention  to  curves  which  are  graphs  of  functions. 

Given  x  6  C2  [a,  6] ,  the  curvature  of  x  at  a  point  t  €  [a,  6]  is  defined  by 


K(x(t))  = 

(  {  )J  (l+i(<)2)3/2- 

(3) 

If  we  use  the  transformation 

ds  =  yj\  +  x(t)2dt , 

(4) 

the  above  functional  (1)  reads 

’  Ja  (l+x(t)2)5/2‘ 

(5) 

By  the  introduction  of  the  “spaces” 


M  =  {x  €  Wi  : 
H  =  {xeW%: 
M+  =  {x  €  M  : 


*(<j)  =  X],  j  =  l,2,...,n}, 
*(fj)  =  3  ~  1)  2,  •  •  • ,  n}, 

*>0}, 


(6) 

(7) 

(8) 


we  can  give  the  problem  the  brief  form: 

minimize  f(x)  given  by  (5)  subject  to  x  €  M+ . 
For  later  reference  we  abbreviate  the  integrand  of  (5)  as 

i(<)2 


F(x)  =  F(t,x,x,x)  = 


(l+i(t)2)5/2- 


(9) 


(10) 


There  is  one  important  subproblem  which  we  term  the  local  problem. 

Local  Problem.  Specify  problem  (9),  but  with  n  =  2  and  with  prescribed 
additional  boundary  conditions 


i(ti)  =  Xi,  i(t  2)  =  X2, 


(11) 


where  ii,  X2  are  any  given  values. 

If  we  could  solve  the  local  problem  satisfactorily,  we  could  already  set  up  an 
algorithm  for  finding  the  solution  of  the  general  problem,  which  in  this  context 
we  term  the  global  problem.  The  algorithm  would  have  the  form: 


Step  1:  Solve  the  unrestricted  problem,  and  call  the  solution  x. 


Step  2:  Replace  the  solution  x  in  those  intervals  I}  =  [t_,,  tJ+i]  in 
which  x  has  negative  values  by  the  corresponding  solution 
of  the  local  problem,  by  using  the  computed  derivatives  at 
the  endpoints  as  prescribed  boundary  conditions. 

Step  3:  Repeat  Steps  1  and  2  by  talcing  all  derivatives  at  the  given 
knots  as  unknowns,  with  the  aim  of  producing  a  C'2-spline. 

It  should  be  noted  that  the  first  two  steps  alone  already  produce  a  C'1-spline. 
For  the  simplified  strain  energy  f*x(t)2dt,  this  algorithm  has  been  proposed 
by  C.  Reinsch  (1988)  and  by  Fischer,  Opfer  and  Puri  (1987).  These  authors 
show  that  the  corresponding  local  solution  can  be  computed  explicitly  in  a  very 
simple  and  efficient  manner.  For  this  simplified  case,  the  algorithm  has  undergone 
extensive  numerical  testing  by  Dauner  (1987)  and  Kroger  (1989). 

The  local  problem  is  a  special  case  of  the  so-called  Z/errmte-problem,  in 
which,  in  addition  to  the  already  stated  conditions,  we  require  that  x(t})  —  ij, 
where  ij,  j  =  1,2, ...  ,n  are  arbitrarily  specified  numbers. 

3.  Necessary  Optimality  Conditions 

If  one  of  the  data  points  should  fall  below  the  x-axis,  then  our  problem  certainly 
has  no  solution.  The  following  assumption  on  the  data  specification  is  therefore 
immediate. 

Assumption  1.  Assume  Af  +  ^  0,  i.e.,  ij  >  0  for  all  j  —  1, 2, . . . ,  n. 

For  our  first  result  we  need  two  more  “spaces”,  the  definition  of  which  de¬ 
pends  only  on  x  €  M + ,  namely 

H?={heH:  x  +  h  >  0 },  (12) 

H?+  =  {h  €  H+  :  x  —  h  >  0  }.  (13) 

The  latter  space  H++  is  usually  referred  to  as  the  envelope  or  hull  of  x,  since  it 
contains  all  those  h  for  which  |/i|  <  x.  It  has  the  properties 

h  6  Ht+  <=>  -he  H?+,  (14a) 

h  e  H;+,x(t)  =  0,  a  <  r  <  6  =$•  h(r)  =  h(r)  =  0  <  h(r).  (14b) 

Theorem  1.  Suppose  that  xo  solves  the  problem  stated  in  (9).  Then 

f'(x0,h)  :=  lim  -{/(xo  +  ah)  -  /(x0)}  =  f  {Fi(x 0)h(t)  +  Fs(x0)Mt)>dt 

a  »  ^  J  n. 


>  0  for  all  h  e  H+o , 

(a) 

=  0  for  all  he  ff++. 

0») 

Proof:  (a)  By  the  definition  of  H+0  we  have  xo  +  ah  6  M+  for  all  0  <  a  <  1. 
Since  in  that  case,  by  definition  of  xo,  we  have  /(xo)  <  f(x o  +  ah),  the  assertion 
follows  from  the  definition  of  the  derivative  of  /. 

(b)  This  follows  from  (a),  by  using  (14a)  in  connection  with  the  linearity  of 
/'(x o,-)  with  respect  to  h.  ■ 


In  order  to  proceed  from  Theorem  1  we  choose  a  fixed  subinterval  Ij  = 
[tj,tj+ 1],  j  =  1,2, . . .  ,n  —  1  and  an  h  €  which  vanishes  outside  Ij.  If  j  >  1 
and  j  +  1  <  n  then  h  £  W%  implies  h^l\tj)  =  =  0  for  £  =  0,1.  By 

applying  partial  integration,  Theorem  1  implies  that  for  ail  j  =  1,2, . . .  ,n  -  1 


J  F(xo)h(t)dt  =  0,  for  h  £  ■H^0+,  and  h(t)  =  Ofor  t  £  Ij, 


where  in  each  interval 


F(X0)  =  Fr(Xo)  -  V, 

V  =  F±(x  o), 


eind 


u(a)  =  v(b)  =  0, 

where  F  is  defined  in  (10).  In  order  to  obtain  (15)  we  use  the  relation 
f  ,+l  Fi(xo)h(t)  =  A(t)u(t)|;>+1  -  fi,+1  v(t)'h(t)dt 

Jtj  3  Jtj 


rh+ 1 
Jti 


v(t)h(t)dt. 


(15) 

(16a) 

(16b) 

(16c) 


If  1  <  j  <  n  —  1  then  h(tj)  =  0,  and  thus  the  above  constant  parts  vanish.  If, 
however,  ;  =  1  or  j  =  n  then  h(t})  may  take  any  value.  In  this  case  we  note  the 
fact  that  v  defined  in  (16b)  is  determined  only  up  to  a  constant,  and  this  allows 
us  to  set  v(a)  —  v(b)  =  0,  at  least  provided  that  we  have  n  >  2. 

For  later  use  we  compute  the  partial  derivatives  of  F: 


— 5xx2 
(1  +x2)7/2’ 


Fi  = 


2x 

(l  +  X2)5/2- 


(17) 


Theorem  2.  (&)  For  n  =  2  the  solution  xo  of  (9)  is  the  linear  interpolant  of  the 
two  data  (t i,*i)  and  (<21*2)-  (b)  For  n  >  2,  if  xo  solves  the  problem  (9)  and  if 
v  is  defined  as  in  (16b)  and  (16c),  then 


(t)  Fi(a )  =  Fi(b)  =  0, 

(it)  Fi  -v  is  linear  in  Ij  =  [t>,f>+i],  provided  x0  >  0  in  I}, 

(tit)  Fi  —  v  is  linear  between  interior  zeros  of  xq,  and  linear  between  interior 
zeros  and  knots,  where  an  interior  zero  of  xq  is  a  zero  in  the  open  interval 

(iv)  ~^{Fi  —  v)  always  has  non-negative  jumps  at  the  interior  zeros  of  xq. 


Proof:  (a)  In  this  case  the  proof  follows  because  xo  £  M+  and  f(x o)  =  0.  (b) 
An  application  of  a  theorem  of  du  Bois-Reymond  tells  us  that  in  each  subinterval 
Ij  the  expression  F  is  a  polynomial  of  degree  1  in  t.  In  addition,  if  r  is  a  zero  of 
xo  then  r  has  to  be  introduced  as  a  new  knot,  since  we  have  h  €  together 

with  the  derivative  h  vanishing  at  r.  The  remaining  details  of  the  proof  follow 
the  corresponding  proofs  in  Opfer  and  Oberle  (1988).  B 

Condition  (i)  corresponds  to  the  so-called  “natural  boundary  conditions”  of 
an  ordinary  spline.  In  the  case  of  an  ordinary  cubic  spline,  condition  («)  is  the 
condition  that  the  second  derivative  should  be  piecewise  linear.  The  remaining 
two  conditions  cover  the  constraints.  Condition  (Hi)  implies  that  any  zero  of  the 
solution  xo  must  be  considered  as  a  new  knot.  The  jump  condition  (iv)  means 
that  the  difference  between  the  right  and  left  derivative  at  a  new  knot  must  always 
be  non-negative. 

Condition  ( ii )  reads  explicitly 


2x(0/(l  +  x(f)2)5/2  -  v(t)  =  At  +  B, 


where 


— 5xx2/(1  +  x2)7/2  =  v. 

Differentiation  with  respect  to  t  yields 

(2(1  +  x2  )x(iii>  -  5xx2 )/( 1  +  x2  )7/2  =  A.  (19) 

If  we  differentiate  once  more  we  obtain  Euler’s  equation 

(1  +  x2){2(1  +  x2)x(lv)  -  20xxx(m)  —  5x3}  -f  35x2x3  =  0.  (20) 

If  x  ^  0  we  may  multiply  equation  (19)  by  x  and  obtain 

d  x2  2xx^***^  5xx3  .. 

. .  — —  --  —  -  -  -  _  —  Ax 

dt(l  +  i2)V2  (1  + *3)5/2  (1  + *2)7/2 

Thus,  by  integrating  we  obtain 

(Ax  +  C)(l  +  x2)5/2  =  x2,  (21) 

which  is  Woodford’s  (1969,  eq.  (5))  equation. 

The  first  three  differential  equations,  valid  in  each  subinterval  and  connected 
by  suitable  boundary  values  can  be  written  as  first  order  systems  which  are  more 
convenient  for  computing.  Equation  (18)  reads 


x  =  u, 


u  =  0.5(1  +  u2f/2(v  +  At  +  B), 
v  =  —  1.25u(l  +  u2)3/2(v  +  At  +  B)2. 
Equation  (19)  takes  the  form 


(18a) 

(18b) 

(18c) 


x  =  u, 

ii  =  V, 


(19a) 

(19b) 


V  =  0.54(1  +  u2)5'2  +  2.5uv2/(1  +  U2).  (19c) 

Finally,  Euler’s  equation  (20)  is  equivalent  to 

x  =  it,  (20a) 

u  =  v,  (20b) 

v  =  w,  (20c) 

w  =  (lOuint;  +  2.5d3)/(1  4-  u2)  -  17.5u2v3/(l  +  u2)2.  (20d) 


In  addition  to  these  equations  we  have  the  boundary  conditions  (16c),  part 
(i)  of  Theorem  2,  the  interpolation  conditions  and  the  smoothness  conditions  at 
the  interior  knots.  Moreover  we  have  to  introduce  new  knots  r,  t}  <  r  <  t}+ 1 
(for  some  ji),  whenever  r  is  an  interior  zero  of  xo-  For  such  points  r,  we  have  the 
additional  conditions  x<j(t)  =  0,  xo(f )  >  0.  The  latter  conditions  reflect  the  fact 
that  xo  has  not  only  a  zero  at  r  but  also  a  minimum. 

The  implied  algorithm  requires  the  repeated  solution  of  one  of  the  above 
sets  of  differential  equations  with  additional  known  and  unknown  boundary  con¬ 
ditions.  Results  in  this  direction  will  be  published  elsewhere. 
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SPLINE  CURVE  FITTING  OF  DIGITIZED  CONTOURS 
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Abstract  This  paper  presents  an  algorithm  for  finding  a  mathematical  curve 
approximation  to  a  digitized  contour.  The  whole  application  consists  in  processing 
optically  scanned  graphic  documents  to  store  them  in  a  data  base  for  consultation  and 
manipulation.  For  each  contour,  an  approximating  parametric  cubic  spline  curve,  with  as 
few  control  points  as  possible,  is  determined  by  minimizing  a  smoothing  criterion.  The 
algorithm  is  particularly  efficient  for  large  data  sets,  the  number  of  control  points  always 
being  very  small.  To  handle  the  curve,  the  B-spline  representation  is  associated  to  a 
hierarchical  data  structure,  obtained  by  generating  the  Bdzier  points  and  subdividing  the 
corresponding  Bezier  segments. 

Key  words  :  Cubic  splines.  Curve  fitting,  Data  compression. 


1.  Introduction 

The  algorithm  presented  is  a  part  of  a  software  project  at  our  school  concerned  with  the 
consultation  and  manipulation  of  documents  archived  in  digital  form.  Documents  which 
were  not  created  in  this  form  are  to  be  converted  to  a  compatible  form.  To  digitize  them, 
we  use  a  scanner  whose  resolution  is  300  dots  per  inch.  The  output,  a  binary  image,  is  an 
array  of  pixels  ( approximately  2400*3500  for  a  A4  document )  without  any  information 
about  the  structure  of  the  document.  For  archiving  purposes,  data  compression  is 
necessary,  and  even  run  length  encoded  forms  such  as  CClTT  group  III  or  IV,  used  for 
facsimile  coding  and  transmitting,  represent  a  very  large  amount  of  data  and  give  no 
information  about  the  structure  of  the  document.  For  black  and  white  graphics  an 


analytical  representation  of  contours  is  useful  for  data  compression,  shape  description 
and  geometrical  transformation.  To  deal  with  complex  documents  it  would  be  necessary 
to  have  preliminary  processing,  which  is  outside  the  scope  of  this  paper,  such  as 
segmentation  into  text,  black  and  white  graphics  and  half  tone  zones  as  described  by  Ito 
and  Saakatani  (1982)  and  determination  of  the  connected  components  of  the  graphic 
zones  as  described  by  Ronse  and  Devijver  (1984).  Since  we  presently  have  studied  only 
very  simple  graphics  including  a  single  component,  a  minimal  basic  preprocessing  is 
necessary  to  determine  an  ordered  list  of  black  pixels  representing  a  contour  or  a  curve 
and  to  convert  it  into  a  continuous  boundary  by  vectorization.  The  next  step  consists  in 
finding  a  good  spline  approximation  by  first  determining  a  small  accurate  set  of  knots  and 
then  adjusting  the  B-spline  coefficients. 


2.  Contour  extraction  and  polygonal  approximation 

Let  us  recall  some  basics  in  binary  image  processing  which  are  given,  for  instance,  by 
Pavlidis  (1982)  : 

-  Background  pixels  (white)  have  value  0  and  object  pixels  (black)  have  value  1. 

-  An  "object"  connected  component  is  a  set  P  of  black  pixels  adjacent  io  an  edge  (D- 
neighbour#  or  to  a  comer  (I-neighbours) . 

A  connected  component  is  entirely  determined  by  its  external  contour  and  possibly  by 
internal  ones  if  there  are  "holes". 

2.1  Contour  extraction 

Definition  :  Let  P  be  a  connected  component  The  contours  C  of  P  are  the  sets  of  adjacent 
pixels  in  P  which  have  at  least  one  D-neighbor  not  in  P. 

A  contour  C  can  be  represented  by  a  closed  path  that  can  be  determined  by  the 
following  algorithm  proposed  by  Pavlidis  (1982)  :  "An  observer  walks  along  pixels 
belonfe.ng  to  the  set  and  selects  the  rightmost  available  until  the  current  pixel  is  the  initial 
pixel".  This  initial  pixel  is  usually  found  by  a  top-to-bottom,  left-to-right  scan.  The 
algorithm  must  be  applied  for  each  contour  of  the  set  P. 

2.2  Line  thinning 

Given  the  scanner  resolution  and  the  graphics  attributes  (pen-size)  a  line  is  scarcely  ever 
one  pixel  wide  and  often  has  interior  pixels.  Since  the  contour  is  adequate  to  describe  a 
filled  shape  but  not  a  line,  it  is  necessary  to  distinguish  shapes  from  lines  that  are 


described  by  the  skeleton  of  the  component  and  its  width.  A  maximum  thickness  T  is  set 
and  the  contour  is  peeled  off  by  removing  all  contours  points  that  are  not  "skeletal", 
following  Pavlidis  (1982).  This  connectivity-preserving  peeling  is  repeated  at  most  T 
times  and,  if  during  this  process  no  points  can  be  removed,  the  skeleton  is  obtained  and 
the  component  is  considered  as  a  line  of  thickness  T'<  T  ;  otherwise  it  is  a  filled  shape. 
Henceforth,  line  skeletons  and  shape  contours  will  be  processed  as  closed  or  open  lines. 

23  Vectorization 

The  next  step  consists  in  converting  the  discrete  pixel-chain  representation  of  lines, 
possibly  closed,  into  a  continuous  very  fine  polygonal  approximation  (vectorization) 
which  vertices  are  the  end  points  of  "straight  line"  segments  of  the  grid.  We  determine 
the  vertices  by  using  the  sequential  method,  based  on  the  minimal  area  deviation  criterion, 
proposed  by  Wall  and  Danielsson  (1984),  with  a  small  tolerance  value  S  : 

Algorithm  1: 

(a)  Let  Mo(xo,y0)  be  the  starting  point  and  Mj  its  contour  successor. 

Initiate  the  algebraic  area  ^  of  (MoMoMi)  to  0.  Let  i  =  2. 

(b)  Calculate  the  algebraic  area  Afjof  (MoM^M;)  and  the  cumulative  area  fi=fi.1-»-Afi. 
Calculate  the  length  L;  from  Mo  to  M;. 

(c)  if  fj  <•  S  Lj  then  increment  i  and  repeat  (b). 

Otherwise  put  in  the  set  of  vertices  and  take  it  as  new  starting  point 
The  detection  of  "characteristic  points"  is  done  simultaneously.  At  any  time  we  keep  the 
segment's  general  direction,  as  one  of  the  8-connectivity,  and  we  mark  the  points  at 
which  a  true  direction  change  (not  temporary  for  a  single  point)  is  found.  These 
"characteristic  points"  are  taken  as  new  starting  points  rather  than  the  last  point  satisfying 
the  criterion . 

3.  Curve  fitting 

We  want  to  determine  a  parametric  curve,  fitted  to  the  ordered  sampled  points  {P,}^ 
obtained  by  the  preprocessing,  which  can  be  written  using  B-spline  basis: 

n 

M(t )  =  Z  Mj  Bi,4(t )  ,  for  te  [To,  Ti  ]  ,  with  the  number  n  of  basis  functions  as 
small  as  possible  and  where  B;  4  are  cubic  normalized  B-splines . 

To  fit  a  parametric  curve  to  a  set  of  points  any  data  point  P;  must  be  assigned 

a  parameter  value  u;  (Uj<u2<...<uN )  that  we  suppose  to  vary  from  0  to  T  for  sake  of 
simplicity.  The  chord-length  parametrization  needs  more  arithmetic  operations  than 


uniform  parametrizadon  but  it  is  more  natural  since  the  points  P;  are  unevenly  spaced  and 
the  contour  of  the  curve  is  very  close  to  the  polyline  joining  the  points  P;. 

3.1  Knots  determination 

To  determine  n  basis  functions  {Bi4}j*=1,  a  subdivision  ti^  t2^...£  tn+4  of  [0,T]  is 
necessary  and  is  automatically  searched  while  keeping  a  trade-off  between  the  closeness 
of  the  fit  and  the  number  of  basis  functions.  There  must  be  as  few  knots  as  possible  but 
the  mean  square  distance  to  the  data  points  must  be  no  greater  than  a  tolerance  level. 
Three  ways  to  obtain  the  knots  placement  have  been  compared; 

-  In  the  first  method,  the  subdivision  {tj}  is  determined  by  using  the  minimal  area 
deviation  criterion  of  Wall  and  Danielsson  (1984),  as  described  above,  with  a  sufficiently 
large  tolerance  value  S. 


Figure  1:  Subdivision  obtained  by  Figure  2:  Subdivision  obtained  by 

algorithm  1  algorithm  1  used  twice 

The  subdivision  {tjjjlj  obtained  by  this  method  is  not  symmetric,  the  "beginning”  and 
the  "end”  of  the  sampling  do  not  play  the  same  part  (fig.l).  To  avoid  this,  it  is  possible  to 
use  the  algorithm  twice,  forward  and  back,  and  to  calculate  the  mean  of  the  two 
subdivisions.  The  subdivision  obtained  by  using  this  algorithm  twice  is  more  regular 
(fig.2). 

-  In  the  second  method,  knots  are  added  until,  on  each  subinterval  I=[T,,T1+.,],  Et  the 
mean  square  distance  of  points  PjG  I  to  the  least-squares  straight  line  is  less  than  a 
parameter  value  E^. 

Algorithm  2 

(a)  Start  with  one  subinterval  [T^TJ  ;  call  M  the  collection  of  subintervals. 

(b)  On  each  Ie  M,  calculate  the  rms  Ej  to  the  least-squares  straight  line. 

(c)  If  Ej  then  split  I  in  two  and  put  both  pieces  into  M,  otherwise  increment  I. 

The  algorithm  is  repeated  from  (b)  until  all  subintervals  are  processed. 

The  subdivision  obtained  with  this  algorithm  (fig.3)  is  not  very  regular,  some  intervals 
seem  to  be  too  small.  This  algorithm  should  be  improved  with  a  "merge"  algorithm  (see 


V 

Pavlidis,1974)  to  merge  two  consecutive  intervals  if  possible.  Split  and  merge  algorithms 
produce  good  results  but  are  time-consuming  since  they  require  multiple  passes  through 
the  data. 

-  The  third  method  is  based  on  least-squares  fitting  by  cubic  polynomials  in  X  and  Y. 
Algorihm  3 

Let  S  be  a  parameter  value. 

(a)  Start  with  one  subinterval  [Tj.TJ;  call  M  the  collection  of  subintervals. 

(b)  On  each  interval  I  €  M,  calculate  2  least- squares  cubic  polynomials  (Px  and  Py) 
for  X  and  Y. 

Calculate  Dx=Ij  (Px  (uj)  -  Xj  )2  and  Dy=Ij  (Py  (Uj)  -  Yj  )*. 

(c)  If  DX>S  or  D  ;>S  then  split  the  interval  in  two  and  put  both  pieces  into  M, 
otherwise  increment  I.  Return  to  (b). 

(d)  Subdivide  each  interval  [Ti,Ti+1]  into  4  to  obtain  the  subdivision  t]<t2^. .  .£  t,,^. 


Figure  3  :  Subdivision  obtained  Figure  4  :  Subdivision  obtained 

by  algorithm  2  by  algorithm  3 

We  use  one  of  these  algorithms  to  obtain  a  subdivision  of  intervals  between  two 
"characteristic"  points:  either  endpoints  or  points  marked  during  the  preprocessing.  After 
obtaining  the  subdivision,  we  add  multiple  (triple)  knots  corresponding  to  marked  points. 
Multiple  knots  are  also  added  at  the  ends  of  an  open  curve  and  cyclic  knots  are  added  if 
the  curve  is  closed. 

In  all  cases,  the  number  of  knots  n+4  is  much  smaller  than  the  number  N  of  data 
points  P|  and  generally  subdivisions  are  quite  similar  (fig.  1  to  4).  Nevertheless  algorithm 
3  should  produce  better  results  in  case  of  lightly  snaky  data.  With  algorithm  1  the 
subdivision  is  not  optimal;  however  this  method  is  attractive  since  it  is  less  time- 
consuming  as  shown  in  Tab.l. 

Table  1:  Time  to  obtain  the  subdivision. 


3.2  Fitting  a  parametric  spline  curve 

The  third  step  of  the  fitting  process  is  to  determine  the  parametric  spline  which  must  be 
close  to  the  points  Pj  and  "smooth".  The  control  points  {Mj}"=1  are  searched  while 
minimizing  the  quadratic  functional: 

N  fT 

h  (M)  =  1  [M(Uj)  -  P/  +  \l  {  [X"]2+  [Y’f  }  dt 
j=l 

o 

The  positive  parameter  |x  controls  the  trade-off  between  the  sum  of  the  distances  of  points 
{Pi}i=1  from  the  curve,  measured  by  the  first  term  of  I2,  and  the  smoothness  of  the 
curve,  measured  by  the  integral,  which  is  a  rough  approximation  to  the  curvature. 
However  this  functional  I2  involves  two  independent  minimization  problems  to  determine 
X  and  Y.  The  solution  is  easily  obtained  (Potier  and  Vercken,  1985)  by  solving  two 
linear  systems  :  Qa=zx  and  Qb=Zy  where  the  matrix  Q  is  a  symmetric  positive  matrix. 
Moreover,  in  the  case  of  open  curves,  Q  is  banded,  whereas  in  the  case  of  closed  curves 
there  are  non-zero  terms  at  opposite  comers,  as  shown  in  figure  5  (in  case  of  quadratic 
splines).  The  solution  of  both  systems  can  be  obtained  by  factorizing  the  matrix  Q  with  a 
data  structure  adaptated  to  the  matrix  (fig. 6)  and  takes  O(n)  operations. 


Figure  5  :  Matrix  Q  in  cyclic  case  Figure  6  :  Cholesky  factorisation 

The  parameter  p  is  adjusted  so  that  the  mean  square  distance  to  the  data  points  is  no 
greater  than  a  value  T  which  may  be  a  number  of  pixels.  In  figures  7  and  8  the  value  of  T 
corresponds  approximately  to  a  tolerance  of  2  pixels.  On  both  figures,  the  initial  curve, 
the  B-spline  approximation  and  the  control  polygon  are  drawn.  The  initial  curve  of  figure 
7  was  designed  with  9  B-splines,  the  digitized  picture  contained  about  5000  pixels  and 
the  skeleton  1139  pixels,  the  vectorization  gave  120  segments.  The  fitting  by  11  B- 
splines,  with  knots  obtained  from  algorithm  1,  is  very  accurate. 


Figure  7  : 1 139  pixels 
and  11  B-splines 


Figure  8  : 781  pixels 
and  20  B-splines 


However  we  did  not  succeed  in  finding  a  good  approximation  with  fewer  basis 
functions.  However,  figure  8  designed  with  16  B-splines,  algorithm  3  gave  better 
subdivision  than  algorithm  1.  The  fitting  with  16  B-splines  kept  the  initial  shape  but 
smoothed  the  comers. 


4.  Interactive  Curve  Handling 


This  smoothing  algorithm  was  integrated  in  an  interactive  graphics  environment  where 
performance  of  the  algorithms  is  of  fundamental  importance.  To  make  geometrical 
transformations  on  a  displayed  curve,  the  user  has  to  identify  the  curve  on  the  screen  with 
a  cursor  pointed  "near"  the  curve.  The  selection  is  done  by  testing  the  convex  hull  of  the 
cubic  B6zier  composite  representation  of  the  curve  obtained  by  the  algorithm  of  Bdhm 
(1981).  By  applying  the  De  Casteljau  algorithm,  each  cubic  B6zier  segment  is  split  in 
two,  if  necessary,  and  this  process  is  recursively  applied.  Successive  splittings  involve  a 
hierarchical  data  representation  of  the  curve  (fig.9). 
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Figure  9:  Hierarchical  data  representation 


The  cubic  Bezier  representation  of  the  curve  can  be  used  for  printing  "nicely"  the  curve 
on  a  PostScript-laserprinter  since  the  PostScript  operator  "curveto"  adds  a  cubic  Bezier 
section  to  die  current  curve. 

5.  Conclusion 

The  fitting  algorithm  presented  in  this  paper  is  very  efficient  and  can  easily  be  inserted  in 
graphic  documents  processing  software.  The  knots  determination,  which  is  crucial,  will 
be  developed  and  the  knot  removal  algorithm  of  Lyche  and  Morken  (1987)  will  be 
compared  with  other  methods. 

Moreover  it  would  be  very  useful  to  extend  this  algorithm  to  more  basis  functions  such 
as  non-uniform  rational  B- splines  to  fit  straight  lines  and  conics. 
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Abstract  Ve  propose  some  straightforward  modifications  to  the  usual 
B- spline  algorithm  for  the  approximation  of  discrete  data.  The  modi¬ 
fications  permit  quasi- interpolation  or  filtering  of  the  data.  We 
propose  a  simple  algorithm  for  the  computation  of  "good”  nodes  -  a 
linear  combination  of  neighbourhood  data,  with  suitable  coefficients 
-  in  order  to  obtain  desirable  properties  for  the  global  approxima¬ 
tion  process.  To  analyse  this  process  -  and  to  determine  the  coeffi¬ 
cients  of  the  linear  combination  -  we  use  the  transfer  function  of 
the  associated  filter,  which  is  a  very  good  tool  to  obtain  a  global 
view  of  the  process  (i.e.,  which  frequencies  of  the  data  are  ampli¬ 
fied,  or  attenuated,  and  how  much,  etc.). 

Kev  words  B-spline  approximation.  Quasi- interpolation.  Filtering, 
Transfer  function 


1 .  Introduction 

Given  discrete  equidistant  data  (x{,yj),  it  is  a  common  require¬ 
ment  to  determine  the  B-spline  approximation  of  those  data,  i.e.  to 
perform  the  following  algorithm: 

FOR  wanted  x, 

compute  cr(x)  —  ^  y(  Bf  (x) 

-  i 

where  the  B,  are  the  usual  cubic  B- splines,  centered  at  x( ;  in  this 
paper  we  will  consider  only  cubic  splines,  but  the  ideas  may  be 
extended  to  any  degree. 

The  important  advantage  of  the  above  computation  is  that  the  curve 
so-obtained  is  very  quickly  determined,  and  follows  the  shape  of  the 


\ 


data  (precisely,  we  have  a(Xj )  -  (yJ  +  1  +  4yj  +  yj  .,,)/6.  But  there  is 
also  an  important  drawback:  such  an  algorithm  allows  no  flexibility, 
in  the  sense  that  we  cannot  force  the  curve  to  be  nearer  to  the  data, 
or,  conversely,  we  cannot  use  it  to  filter  noise  which  may  be  present 
in  the  data. 

To  overcome  this  drawback,  we  propose  splitting  up  the  nodes,  Y?  , 
of  the  B-spline  approximation,  from  the  data  y^ ;  for  fast  computation, 
we  will  require  Y?  to  be  a  linear  combination  of  2n+l  of  the 
placed  around  y{  (the  coefficients  of  the  linear  combination,  as  well 
as  the  value  of  n,  being  chosen  according  to  the  type  of  result  to  be 
obtained,  see  later).  So,  we  propose  performing  the  following 
algorithm: 

For  concerned  i 

j  compute  Y?  -  £  oc^y, .  j  . 

j— n 

For  wanted  x 

I  compute  o" (x)  -  ^  Y?  B, (x) . 

I -  iGZ 

The  aim  of  this  paper  is  to  show  some  way  of  choosing  appropriate 
coefficients  of  the  linear  combination,  and,  in  order  to  do  so,  to 
present  and  use  the  "transfer  function"  (which  is  also  called  "atte¬ 
nuation  function*  in  some  papers) ,  which  is  a  very  convenient  and 
comprehensive  tool  to  obtain  a  global  view  of  the  approximation 
process . 

To  do  so,  we  will  suppose  we  have  an  infinite  regularly  spaced 
data  set  (it  is  easy,  if  necessary,  to  extend  a  finite  set  to  an 
infinite  one,  in  an  appropriate  way),  so  V  i  6  Z,  x,  -  ih.  Also,  we 
will  take  symmetric  combinations  (V  j  £  [1,  n] ,  a?j  -  a"),  so  we  will 

n 

use  the  equivalent  form  Y?  -  ^bJ  (62^y)j  ,  where 

j-° 

<«2y>i  “  Yfi  '  2y,  +  y,.,  • 


2.  Tranfer  function 
2.1  Definition 

Presentation  The  first  idea  is  to  study  only  the  result  of  the  appro¬ 
ximation  scheme  at  the  data:  we  will  look  only  at  (on(xj))1.  e  £,  and 
not  at  </*  (x)  for  x  *  x, .  In  other  words,  we  will  study  the  transfor¬ 
mation  Tn:  y  €  F?  — ►  z  €  defined  by 


v  j  6  Z,  Zj  -  o"(Xj.)  -  2  S  bk&2lty  BCXj.j). 

i6Z  Uc— n  J, 

Since  the  values  a(xf )  provide  information  on  ct(x) ,  studying  this 
discrete  transformation  will  provide  knowledge  of  the  whole  approxi¬ 
mation  process. 

Obviously  Tn  is  a  linear  transformation,  and  the  vector 
y  -  ea  -  (e2l1tak)kgr  (here  i2  -  -1)  is  an  eigenvector  of  Tn  . 

Definition  Let  us  call  Hn (Ct)  the  eigenvalue  of  the  above  transforma¬ 
tion  Tn ,  associated  with  the  eigenvector  ea.  The  application 
H,, :  a  — »  ^(a)  is  called  the  "transfer  function"  of  the  filter  Tn . 

Hj,  is  a  real-valued  function  which  is  even  and  periodic  (of  period  1/h) . 

Remarks:  By  definition,  ^(a)  is  the  coefficient  of  amplification  of  a 
(co) sinusoidal  signal  of  frequency  a.  ^(0)  is  the  coefficient  of 
amplification  of  constants  (we  will  always  require  Hn(0)-1  !). 

As  (eot)ot  e  p  are  the  only  eigenvectors  of  Tn ,  it  is  understandable 
that  the  set  of  eigenvalues  (^(a))^  6  -  i.e.  the  function  Hn-  is 

indicative  of  the  whole  transformation  Tn . 

As  a  particular  case,  if  V  a  6  R,  H„(oO  -  1,  then  Tn  is  the  identity. 

2.2.  Use  of  transfer  function 

Evaluation  of  Hn:  theorem 

Let  Hn  be  defined  as  above . 

Let  b"+1  -  0  .  Then 

n+1 

H^a)  -  b|J  +  £(bj  +  bJ.1/6)(-4sin2TOh)k; 
k-1 

The  proof  is  quite  easy:  since  V  y  €  R?, 
n+1 

Tn(y)  -  b{Jy  +  £  (bj  +  bk.1/6)62ky;  so  if  g  is  the  elementary  filter 
k-1 

y  — ►  S2ky  (whose  transfer  function  is  G(a)  -  -4sin2ioh), 
n+1  n+1 

Tn  “  bo8°  +  H  <bk  +  bk-i/6)gk,  and  so  H„  -  bj  +  £  (bj  +  bJ.1/6)Gk. 
k-1  k-1 

Error  majorat  ion:  theorem 

Let  g  and  h  be  two  filters,  with  respective  transfer  functions 
G  and  H. 

g  b 

— ►  z  and  y  — *  z '  . 


Let  y 


For  any  periodic  function  f  of  period  P,  let  lfl2  -  JJp(f(x))2dx. 
If  ye  f2,  Iz-z'l  Iz-z'l  2  <  IG-Hlorlyl  2. 

If  y  6  41 ,  lz-z'1  Iz-z'l  2  <  BG-Hlj . lyl  , . 


3.  Choosing  the  coefficients 

Of  course,  the  actual  values  of  bj  will  determine  the  properties  of 
Tn ,  and  so  of  the  approximation  process .  The  aim  of  this  section  is 
to  present  some  criteria  in  order  to  determine  interesting  values  of 
b " .  To  do  so,  having  chosen  a  transfer  function  G,  we  want  to  deter¬ 
mine  b"  so  that  the  associated  function  is  as  close  to  G  as  pos¬ 
sible  (if  we  want  to  quasi -interpolate  data,  we  choose  G  5  1,  if  we 
want  to  filter  data  as  much  as  possible,  we  will  choose  G(0)  -  1, 
G(a)  -  0  for  a  *  0) . 


3.1.  First  criterion:  truncated  development 


Definition  The  b"  are  chosen  in  order  that  the  first  terms  of  the 
limited  development  at  some  point  Otg  (for  example  0Cq  -  0)  of  Hn 
coincide  with  those  of  G;  so,  if  G  is  written  as 


G(ot)  -  ck(-4  sin2itth)k  ,  we  obtain  (otg  -  0): 

k£N 

b3  -  ^0 

-  j  -  l,...,n  :  b"  -  Cj  bJ.,/6 

j>n+l  :  b  ,  -  0 

* 


For  quasi -interpolating  data  (G  =  1)  ,  we  obtain 

Vk  £  [0,n],  b£  -  (-l/6)k 

For  a  high-cut  filter  (G(0)  -  1,  Vk  6  N,  G(k>(l/2h) 
Vk£  [0,n]  ,  b£  -  I  (l/4)k  (~Vk£  [0,n]  .  aj 


0) ,  we  obtain: 

t]<l/4)l)- 


Error  majorat  ion:  theorem 
Let  n  £  W 

Let  Vj  £  [0,n]  ,  b?  -  (-1/6)'  ; 


Let  ^(x) 


i  (i»c‘z 

i£Z  U-0 


\ 


/ 


B(x 


Vj  G  [0,n],  b? 
-  x,). 


0. 


Then  Vj  £  Z.  on( Xj )  -  y}  -  (-l/6)n+1 (S2n+2y)j . 

As  a  consequence  majorations  of  lon(xJ  )  -  y-}  I  may  easily  be  obtai¬ 
ned,  and,  in  particular,  o'1  interpolates  any  polynomial  of  degree  at 
most  2n+-l. 

Proof:  induction  on  n. 

Figures:  Every  figure  is  provided  for  n  -  0,  1,  2,  3,  4  and  5. 

Figures  1  and  2  are  transfer  functions  of  a  quasi- interpolation  and 
of  a  high-cut  filter. 

Figures  3  and  4  are  o"  for  some  data:  (the  o  are  the  data;  the  +  are 
the  computed  nodes  Y?).  Figure  3  is  for  step  data:  i<0  —*  y{  -  0, 
i>l  — <*  yf  -  1.  Figure  4  is  for  a  parametric  curve:  each  component  is 
calculated  separately  (equidistant  parameter) . 

3.2.  Second  criterion:  least  squares 

The  truncated  development  criterion  is  very  interesting  as  the 
formulae  obtained  are  very  simple  and  lead  to  a  many  interesting 
theorical  and  practical  results.  But  other  criteria  give  better 
results  in  most  cases.  An  example  of  such  a  criterion  is  given  in 
this  section. 

3.2.1.  Criterion 

Looking  at  theorem  2.3,  the  idea  now  is  to  minimize  IIH,,  -  Gllz  .  This 
is  quite  simple  as  En  -  J*o^h(G(a)  -  Hn(a))zda  is  a  quadratic  func¬ 
tion  of  the  coefficients  b" .  Minimising  En  leads  to  the  solution  of  a 
linear  system  with  unknowns  b" . 

Of  course,  we  can  add  some  additional  constraints,  such  as,  for 
example : 

-  reproducing  constants:  1^(0)  -  1  («“+  bg  -  1,  b"  -  -1/6) 

-  reproducing  polynomials  of  degree  <  3:  Hn(0)  -  1;  H"n(0)  -  0 

(*+  bg  -  1  ,  b$  -  -1/6  ) 

-  no  frequency  amplified:  Wa  6  R  ,  Hn(a)  <  1 

-  no  frequency  inversed:  \fa  £  R  ,  ^ (a)  >  0. 

Here  are  some  values  obtained  for  quasi  interpolation  (i.e. 

G(a)  =  1),  with  the  reproducing  constants  constraint. 


n-1 

bj-l 

b{  — 8/23 

n-2 

d— 474 

b|-l 

bf— 30/d 

b|-47/d 

n-3 

d-8838 

H 

i 

mo 

.£> 

b^— 1890/d 

b|  — 351/d 

b^  — 240/d 

n-4 

d— 15443 

b*-l 

b}  —  22776/d 

bJ-11320/d 

bj -4002/d  bJ-1137/d 

Figure  3  Figure 


We  notice  that,  for  n  -  1,  with  ctg  -  (l/‘nh)sin' 1  (5/8)o;(l/h)0.215, 
we  have  MaxCH^a))  -  H1  (ocq  >  1.1415  ,  which  may  be  considered  as 

o0=R 

amplifying  too  much  the  frequencies  around  0£q  .  So  we  suggest  to  use, 
for  n  -  1,  b,  -  -1/4  which  lies  between  the  truncated  development 
case  and  the  least  squares  one,  and  gives  improved  results. 

3.2.2.  Figures : 

Figures  5  to  8  are  for  quasi -interpolating  B- spline  approximation, 
with  the  least  squares  criterion,  each  one  for  n  -  1,  2,  3,  4  and  5 
(for  n  -  1,  b,  -  -1/4) 

Figure  5:  transfer  function  H^a) 

Figure  6:  transfer  function  H„(o)  (enlarging  the  scale  around  ^(a)). 
Figures  7  and  8:  response  to  the  same  data  as  figures  3  and  4. 

When  comparing  figures  5-8  with  figures  1-4,  we  can  see  how  much 
closer  the  least  squares  B-spline  approximation  is,  for  the  same  n, 
to  the  interpolant  than  the  truncated  development  B-spline  approxima¬ 
tion  is. 

3.3  A  pretty  filter 

It  is  worthwhile  to  mention  the  filter  defined  by  its  a"  coeffi¬ 
cients  (see  section  1)  by:  Vj  €  [-n,n],  a"  -  (n  +  1  -  ljl)/(n  +  l)2 . 
For  lack  of  space,  it  is  not  possible  to  detail  its  properties,  but  we 
can  say  that  it  is  a  very  efficient  and  simple  high-cut  filter.  Its 
transfer  function  is  shown  at  figure  9,  for  n  -  5,  15,  45;  the  curve 
obtained  by  filtering  some  noisy  data  (shares  of  some  French  company) 
is  shown  in  figure  10. 


4.  Conclusion 

4.1.  "Transfer  function"  is  a  most  efficient  way  to  obtain  a  global 
view  of  an  approximation  process.  For  quasi- interpolation,  it  is  much 
better  than  "an  order  of  convergence  h9"  (or  "interpolating 
polynomials  of  degree  at  most  q  -  1"). 

4.2.  We  think  that  this  algorithm  may  have  many  applications: 

-  in  CAGD:  thanks  to  this  algorithm,  an  operator  has  no  need  to  give 
points  far  from  the  desired  curve,  which  is  much  more  convenient 

-  for  closed  curves:  the  way  to  extend  the  points  in  a  periodic  way 
is  obvious,  and  there  is  of  course  no  linear  system  to  solve. 

-  for  filtering  data:  most  methods  need  to  solve  large  linear 
systems;  here  we  get  satisfactory  results  with  little  computation. 

-  for  d-dimensional  surfaces  obtained  by  tensor  product:  simple 


Figure  7  Figure 


quasi- interpolation  is  very  interesting  as  the  usual  tensor  product 
B-spline  approximation  smoothes  the  data  in  a  quite  important  way, 
which  of  course  is  not  the  case  of  tensor  product  B-spline  quasi¬ 
interpolation. 

4.3.  This  method  is  very  easy  to  introduce  in  existing  algorithms:  we 
only  have  to  add  a  line  to  compute  the  nodes  Y? ;  everything  else 
remains  unchanged! 

4.4.  The  "modified  B- splines"  Bn  can  certainly  be  used  in  place  of 
the  usual  B-splines  in  every  application  (as  rational  splines).  So, 
we  can  have  the  opportunity  of  one  (or  even  many)  degree  of  freedom. 

4.5.  At  present  work  is  carried  out  for  determining  best  high-cut 
filter  by  cross-validation  techniques.  Extensions  are  also  being  made 
to  thin  plate  splines. 
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ON  KNOTS  AND  NODES  FOR  SPLINE  INTERPOLATION 


PHILIP  W.  SMITH 
IMSL 


Abstract  This  paper  explores  the  relationships  among  the  knots,  nodes,  and  norm 
of  the  interpolating  spline  projection.  In  particular,  we  derive  lower  bounds  for  the 
norm  of  the  projection  in  terms  of  the  knots  and  nodes.  We  then  turn  our  atten¬ 
tion  to  the  rede  played  by  equioscillating  splines  in  deriving  the  best  interpolating 
projector  for  a  fixed  spline  subspace. 

Keywords:  equioscillating  spline,  B-spline,  projection. 


1.  Introduction 

The  general  problem  of  choosing  knots  and/or  nodes  for  spline  interpolation  is  ad¬ 
dressed  in  this  paper.  Controlling  the  size  of  the  interpolating  projection  is  the  crite¬ 
rion  for  ‘good’  knots  and/or  nodes.  We  adhere  to  the  following  notation  throughout 
this  work.  The  spline  subspace  Sf,  t  :=  <  •  •  •  <  <„+*,  i®  linear  span  of  the  n 

normalized  B-splines  . . . ,  !?„,*,«  where 

Bi,k,t(x)  :=  {U+k  —  ^«)[^*>  •  •  •  —  ®)+  *• 

If  no  misunderstanding  can  arise,  we  denote  by  B{.  Let  r  :=  rx  <  •  •  •  <  r„  be 
n  nodes  of  interpolation.  The  operator 


P  ■  C[tutn+k]  -  5* 

is  well  defined  by 

(Pf)(ri)  =  f(ri)  i  -  l,...,n 

if  and  only  if  jt  0  (i.e.  usually  U  <  however,  equality  can  occur  if  k 

knots  are  stacked  together).  Throughout,  we  assume  that  P  is  well  defined. 


The  central  issue  is  the  relationship  of  the  norm  of  the  operator  P,  ||P||,  to  the 
knots  and  nodes.  Instead  of  working  directly  with  P,  we  shift  our  attention  to  the 
finite  dimensional  linear  operator  A  :  Rn  — ►  Rn  defined  by 

Aij  =  (An,T,t)*j  =  1  ^  *,J  $  n. 

The  relationship  between  P  and  A  is 

Pf  = 

i=i 

where  /(r)  :=  (/(Ti),...,/(rn)).  Furthermore,  one  observes  [1]  that 

(1) 

where  all  norms  in  this  paper  are  supremum  norms  on  the  appropriate  space.  The 
important  feature  of  the  above  inequality  is  that  it  is  independent  of  n  or  t. 

Recall  that  A  is  totally  positive.  This  means  that  A-1  is  checkerboard,  that  is 

(-ly+M-1  >  0, 

from  which  it  follows  that 

||A-l||  =  IM-1«|| 

where  e<  =  (-1)'”1,  for  *  =  1, . . . ,  n.  From  the  inequality  (1),  we  see  that  upper  and 
lower  bounds  on  || A-1 1|  in  terms  of  t  and  r  yield  similar  bounds  on  ||P||. 

We  focus  on  two  problems.  The  first  problem  is  fundamental  but  extremely 
difficult:  Given  k  and  r,  choose  the  ‘best’  knots  for  interpolation.  We  interpret  this 
to  mean,  find  knots  t“  which  satisfy 

U?'\\  <  II A-1  II 

among  all  competing  knot  sequences  t.  Perhaps  it  would  be  even  more  practical  to 
find  ‘acceptable’ t**  which  depend  on  local  information  and  satisfy 

«»IW.||  <  iMr-'H 

for  some  fixed  positive  constant 

The  second  problem  is,  in  some  sense,  the  dual  problem:  Given  k  and  t,  choose 
the  best  r  for  interpolation.  We  interpret  this  to  mean,  find  nodes  r*  so  that 

U;}\\  <  ll^ii 

among  all  competing  nodes  r.  S.  Demko  [3]  has  shown  that  there  exists  r**  so  that 

m;~ii  < 

In  fact,  it  has  been  noted  by  Morken  [4]  that  Demko’s  choice  of  nodes  yields  the 
best  set  of  nodes  for  the  given  knot  sequence. 


» 


» 


2.  Estimates  for  HA-1)! 

In  this  section,  we  elaborate  on  a  technique  for  obtaining  lower  estimates  for  ||A-1|| 
that  first  appeared  in  [1].  Recall  [2],  that  if  s  €  S*  and 

j 

then  (ignoring  boundary  effects) 

j  tj+k-i  — l3 

We  define  the  weighted  difference  operator  v[^  by 

:=  (t  -  I)'’''''1-' 

tj+k-l  ~  *} 

which  allows  us  to  rewrite  s'  as 

•'  =  £ v'ShBjj,-,. 

3 

More  generally,  if  we  take  the  rth  derivative  then 

=  £  vW»,a,»-r. 

3 

where  we  define  vj*2  recursively  by 

y(r)  _  yO)  y(r_1) 

The  goal  in  this  section  is  to  estimate  the  coefficients  of  an  oscillating  spline  and 
to  relate  this  information  to  bounds  on  ||i4-1||.  If  the  spline  a  satisfies  s(ry)  =  ( — 1)*— 1 
for  i  =  1, . . .  ,n,  then  the  /<*>  norm  of  the  coefficients  is  ||A~1||- 
Theorem  1.  Let  Jfc,  n,  t,  and  r  be  given.  We  assume  that 


A  An,t,T 

is  invertible  and  that  5*  6  L£o[Ti,rn].  Set  e(i)  :=  (-1)’-1  for  t  =  1, . . .  ,n.  Then 

||A-i||  >  !~  v’r*+,:~r! 

**i,r 

where  i  =  1, . . . ,  n  -  r  and 

A,>  :=  maxdvljejj  :  (fj,f;+*_r)n(r,,r,+r)  ^  0}. 

Proof:  Let  a  €  5*  satisfy 

»(n)  =  e,  =  (  — 1),_1  »  =  l,...,n. 


•  • 


i 


Then  a  =  J2j  ajBj,k  and  Aa  =  e.  It  follows  that  ||  A- 1 1|  =  ||a||.  From  the  Peano 
kernel  theorem  we  have 


[ri,  • . . ,  TV^.r]s  —  [r, , . . . ,  Ti+r]e  £ 


H^(r)  11[t, 


»T*+ 


r! 


Now 


*(r)  = 


and  hence 


ll^llln, r*r]  <  max{lvaail  :  supp  Bjik.r  n  (r,-,rt+r)  ^  0}. 

Since  the  matrix  A  is  variation  diminishing,  we  know  that  ctj(— 1)J_1  >  0  for  all 
j.  It  follows  that  there  is  no  cancellation  in  the  differences  vj^aj,  and  hence 

|Vj2«jl  <  [vM«y|||«||. 


Assembling  the  pieces  yields 

|[r,v..,r<+r]e|<||«||^, 

completing  the  proof. 

We  mention  two  corollaries  of  this  result. 

Corollary  2.  (de  Boor  [1])  Let  r  =  1,  then 


IIA-MI  > 


dj 

(Ti+l  ~  Ti)(k  -  1) 


and 


where 


\\P\\  > 


_ Dkdj _ 

(r,+i  -  r, )(*-!)' 


<k  :=  min{tj+k_i  -  tj  :  n  (tv,t,+1)  ^  0} 


The  second  inequality  was  derived  by  Ching-Ching  Rojas,  a  student  of  ours. 
Corollary  8.  Let  r  =  2  and  fc  =  3  with  5*  C  L2X .  Then 


II  A"1 1|  > 


_ _ 

(n+2  -  n+i)(rl+i  -  Ti) 


where 


Si  =  max 


(b+i  ~  *7-i )  +  (*j+a  -  h) 


(tj,tj+i)n(ri  -  r,+2)}. 


i  i 


3.  Equioscillating  Splines 

In  [3]  it  was  shown  that  there  exists  a  spline  T  =  £  a’jBhk  €  5*  which  equioscillates 
maximally.  That  is,  on  [ti,tn+fc]  there  exist  n  points  n  <  r2  <  •  *  •  <  r„  so  that 

Tin)  =  (-1)-1 

Ill’ll  =  1- 

Combining  this  observation  with  the  inequality 

^vn<imi~i 

yields  the  following  result 

Theorem  4 .  (Demko  [3])  Given  n,  t,  and  k,  there  exist  nodes  r  so  that  the 
matrix  A  :=  Anit,T  satisfies 

IIA-1!!  <  Dk. 


It  is  important  to  note  that  this  result  yields  a  universal  upper  bound  (depending 
only  on  Jfc)  for  these  particular  nodes.  Indeed,  this  result  illustrates  a  dichotomy 
between  the  dual  problems  of  obtaining 

(A)  sup  inf  ||  || 

(B)  supinf||At-‘|| 

T  * 

Note  that  by  Theorem  3.1,  (A)  is  bounded  above  by  Dk,  while  (B)  is  always  infinite 
by  Theorem  2.1  or  Corollary  2.2,  if  it  >  2  and  n  is  sufficiently  large. 

As  Morken  [4]  pointed  out,  the  abscissae  of  the  extreme  points  of  the  equioscil- 
lating  spline  yield  the  best  points  for  interpolation.  This  can  be  seen  by  setting  T 
to  be  the  equioscillating  spline  with  equioscillation  points  r,  as  above,  and  letting 
Cx  <  • '  •  <  Cn  be  another  set  of  nodes.  Let  s  €  5*  be  defined  by  s(Q)  =  (-1)*-1, 
then 

t  = 

*  =  EM*;,* 

(s-T)(C,)(-l)’-l>0. 

This  means  that  the  coefficients  of  s  -  T,  which  are  bj  -  a*,  weakly  oscillate  (i.e. 
( bj  -  ajX-iy-1  >  0).  But  since  bj  and  a*  oscillate  in  the  same  orientation,  we 
conclude  that 


l«;i  <  \bj\  j=  l,...,n 
hence  ||o*||  <  ||6|| 
or  || A;1 1|  <  l|A(-‘|l 


We  now  discuss  the  computing  of  the  nodes  r  for  the  equiosciilating  spline. 
First,  observe  that  T  (and  hence  r)  is  unique,  since  T  can  be  viewed  as  an  error  in 
the  best  approximation  of  a\Bi,k  from  the  span  of  {B2,*, .  •  • , £„,*}.  Furthermore,  if 
we  set  ti  =  •  •  -  =  4  and  4+1  =  •  •  •  =  t„+k,  then  Ti  =  4  and  r„  =  tn+i-  This  follows 
by  noting  that  all  B-splines  vanish  at  4  except  for  Bj  *,  and  this  spline  takes  on  its 
maximum  at  4-  Similarly,  all  B-splines  vanish  at  4+1  except  which  takes  on 
its  maximum  at  4+1  • 

The  computational  algorithm  is  a  Remez  exchange  algorithm,  where  r1  is  chosen 
so  that  At  T\  is  invertible  and  r}  =  4,  Tn  =  4+i-  Currently,  we  choose  r  to  be  the 
interior  knot  averages  for  the  B-splines,  that  is 


Ti 


t+l 

k- 1  ■ 


We  then  compute  which  satisfies 


s1(r/)  =  e,  =  (-l)‘-1. 


We  then  choose  r2  to  be  the  unique  strictly  increasing  vector  satisfying 

ri  =  4 
rn=  4+1 

si(Tj)—  0  j  =  2, —  1. 

Then  r 2  is  substituted  for  r1,  and  the  process  is  repeated  until  convergence. 
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2.  Polynomial  &  Piecewise  Polynomial  Approximation 


A  BASIS  FOR  CERTAIN  SPACES  OF  MULTIVARIATE  POLYNOMIALS  AND 
EXPONENTIALS 


W.  DAHMEN 

Fachbereich  Mathematik 
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Abstract  A  basis  for  certain  spaces  of  multivariate  polynomials  and 
exponentials  is  constructed  from  the  polynomial  pieces  of  so-called 
multivariate  truncated  powers  introduced  by  Dahmen  (1980).  The  need 
for  such  a  basis  arises  in  connection  with  certain  Hermite  interpo¬ 
lation  problems  which  were  recently  investigated  by  Dyn  and 
Ron  (1988a,  b) .  It  is  indicated  how  to  compute  the  elements  of  the 
basis  explicitly  by  means  of  various  representations  of  truncated 
powers . 

Key  words:  Truncated  powers,  Exponential  splines.  Interpolation, 
Construction  of  bases.  Common  null  space  of  families  of  differential 
operators . 


1 .  Introduction 

The  literature  of  the  past  few  years  reflects  a  rapidly  growing 
interest  in  multivariate  interpolation  problems.  The  numerous  methods 
which  have  been  developed  so  far  are  based  on  a  diversity  of  function 
systems  where  an  important  role  is  played,  of  course,  by  spaces  of 
splines  and  polynomials. 

This  paper  is  concerned  with  certain  spaces  of  multivariate  poly- 


nomials  and  exponentials  which  are  obtained  as  common  null  spaces  of 
certain  differential  operators  (see  e.g.  Dahmen  and  Micchelli  (1983a), 
(1985),  (1987)).  These  spaces  play  a  central  role  in  the  theory  of 
multivariate  splines  on  regular  grids  as  well  as  in  various  related 
combinatorial  and  algebraic  aspects  (cf.  Dahmen  and  Micchelli  (1968)). 
Moreover,  they  arise  in  connection  with  certain  Hermite  interpolation 
problems  that  were  recently  investigated  by  Dyn  and  Ron  (1988a, 

1988b) . 

However,  the  practical  solution  of  such  interpolation  problems  would 
require  the  explicit  knowledge  of  bases  for  these  spaces  which  it  seems 
have  not  previously  been  available. 

The  objective  of  this  paper  is  to  construct  such  bases  by  making 
use  of  the  intimate  relationship  to  the  corresponding  multivariate 
splines.  More  precisely,  it  will  be  shown  how  to  obtain  a  basis  from 
the  pieces  of  certain  splines  called  multivariate  truncated  powers  in¬ 
troduced  by  Dahmen  (1980).  The  computation  of  these  pieces  is  facili¬ 
tated  by  various  known  representations  for  truncated  powers . 

The  paper  is  organized  as  follows.  In  Section  2  we  state  the  rele¬ 
vant  definitions  and  collect  some  background  material.  Section  3  is 
devoted  to  the  general  construction  of  bases  for  the  above  mentioned 
spaces.  Finally,  an  example  is  discussed  in  Section  4. 

2.  Some  definitions  and  background  material 

Let  us  start  by  recalling  the  definitions  and  some  properties  of  certain 
multivariate  (polynomial  and  exponential)  splines  which  will  play  a 
crucial  role  in  subsequent  discussions.  For  more  technical  details  and 
proofs  the  reader  is  referred  to  Dahmen  (1980),  Dahmen  and  Micchelli 
(1983b),  (1987).  To  this  end,  let  X  =  {x^,...,xn}  be  a  set  of  (not 
necessarily  distinct)  vectors  inlRS^{0}  and  let  4  £  Cn  be  fixed.  For 
convenience  the  (s *  n) -matrix  whose  columns  are  the  elements  of  X  is 
also  denoted  by  X .  Under  suitable  assumptions  on  4  and  X  one  can 
then  define  a  function  T  (*jx)  by  requiring  that 


f  f (x) T  (xix)dx  =  /  e“U  f(Xv)dv  (1) 

'  s  4  Jn 

+ 

holds  for  all  f  £  C„QRS)  (where  v  >  z  =  v,z,  +...+  v  z  denotes  the 
0  linn 

standard  inner  product  on  t  ).  A  case  of  particular  interest  is  4  =  0. 
For  T( • | X) :=  Tq ( • | X)  to  be  well  defined  one  has  to  assume  that 


0  Z  [x]  , 


where  [x]  denotes  the  convex  hull  of  the  set  X  .  If  <X>,the  span  of  X  1 
is  all  of  1RS  ,  T( • | X)  is  easily  seen  to  be  indeed  a  function  with  ' 

support 


supp  T ( • | X)  =  <X>+  , 


where  <X>+  =  {Xv  :  v  €  is  the  cone  spanned  by  X.  T(*|x)  can  be 

shown  to  be  a  piecewise  polynomial  of  degree  at  most  n  -  s, where  the 
polynomial  pieces  are  separated  by  the  hyperplanes  in 


C  (X)  =  {<V>  :  V  £  8  .  (X)  }  . 

s- 1 


8a(X)  =  {V  c  X  :  dim<V>  =  jv|  =  U 


and  I V |  denotes  the  cardinality  of  V.  In  general,  when  4  is  some 
complex  vector  one  has  to  assume 


Re  4.  >  0 
j 


whenever  xJ  belongs  to  X',the  largest  subset  of  X  such  that  there 
exists  6£Z|  1 ,  S>0,i=l,...,|x'|  with  X'B  =  0.  Again 

T  ( • | X)  turns  out  to  be  piecewise  analytic  with  cut  regions  given  by 
(4)  and  support  <X>+.  In  general  its  pieces  are  composed  of  poly¬ 
nomials  and  exponentials  to  be  explained  in  more  detail  below. 
Specifically,  one  readily  concludes  from  (1)  that  for  n  =  |x|  =  s. 


<X>  =  B  , 

T  (x|x)  =  x<x>  (x)|det  X | -1e  (X  x)  .  (6) 

In  general  T^(*|x)  can  be  evaluated  recursively.  In  addition, 
explicit  analytic  expressions  are  available  for  special  choices  of  X.  4 
The  following  case  will  be  needed  in  the  next  section.  To  this  end,  let: 

-t . 

.  4 . 

r  1  1  S  S  i  4 

x  =  (y  ,...,y  ,...,y  ,...,y  }  .  :  , 

i  4  * 

|  T£ 

i  i  2  S  i  *5  { 

where  each  y  occurs  m.  times  and  Y  =  {y  _ _ ,y  }  spans  ]R  .  Denoting  _ 

-1  1 

by  u  =  Y  x  the  coordinates  of  x  with  respect  to  the  basis  Y  one  has  2‘ 
(cf.  Dahmen  and  Micchelli  (1983b))  ‘ 

m^-1  m  -1  ; 

T (x | X)  =  X<y>  (x)u1  . ..u_S  •  (7) 

+ 

When  p  t  0  note  first  that  for  any  nonsingular  (s  x  s)  -matrix  A: 


|  det  A|T^(Ax|AX)  =  T^(x|X). 


Choosing  A  =  Y  one  then  easily  verifies  that 


T  (x  X)  =  I  det  Y  T  (u  Y  X)  =  IT  T  j  (u .  I  1  (j)  )  ,  (8) 

U  U  3  =  1  M  3  1 

where  n"1  =  (4  .  ,  —  ,n  )  and  l(j)  denotes  the  (lxm.)- 

1  3-1  1  3  J 

matrix  with  all  entries  equal  to  1.  Moreover,  noting  that  the  Laplace 

i  in .  _  i 

transform  of  each  factor  T  j(*|l(j))  is  given  by  (u  l  +  X)  , 

X  >  -  4  3  •  i  =  1»  —  ,m^  /  each  univariate  factor  in  (8)  is  a  con¬ 
volution  of  the  functions  t^  e  Defining  D  f  =  ES  .  y .  and 

+  n-1  y  :=1  3  3xi 

denoting  by  the  element  in  t  obtained  from  u  by 

discarding  it  is  also  not  hard  to  show  that 


(tij  +Dxj)Ti(-|x)  - 


(9) 


Many  of  the  properties  stated  above  indeed  follow  from  (9) .  In  parti¬ 
cular,  setting 

V(X)  =  {V  <=  X  :  <X^V>  t  IRS}, 

a  repeated  application  of  (9)  yields, in  view  of  (6),  that 

(IIv€V^v  +  DvnVX'X)  =  0  (10) 

for  all  V  €  y(X),  provided  that  x  does  not  belong  to  any  of  the  cut 
regions  in  (4).  Hence  T  (• |x)  belongs  locally  to  the  space 

D  (X)  =  tf  €  V  ’  (JRS)  :  (Iiv€v(nv  +  Dv))f=0,Vv  €  V  (X)  }  . 

s  s 

where  V’ OR  )  denotes  the  space  of  Schwartz  distributions  on  ]R  .  The 

fact  that  D^(X)  is  finite  dimensional  is  a  consequence  of  the  follo¬ 
wing  more  general  result. 

Theorem  1  Suppose  that  for  X  as  above  {Lv^v£x  is  a  family  of  commu¬ 
ting  endomorphisms  on  some  linear  space  S  .  Setting 

K (X)  =  (f  £  S  :  (n  _  L  )f  =  0  ,  V  €  V(X) } , 
v€V  v 

one  has 

dim  K (X)  <  Iy€g  {x)  dim  K (Y)  .  (11) 

s 

Theorem  1  is  a  special  case  of  Theorem  3.1  in  Dahmen  and  Micchelli 
(1987) .  Moreover,  conditions  are  given  there  which  ensure  that  equality 
holds  in  (11).  Specializing  these  facts  to  the  case  at  hand  gives 

dim  D  (X)  =  |8  (X)  | 

U  1  s 

independent  of  H  f  t"  . 


(12) 


(13) 


i,  i 

Defining  for  Y  =  {x  . x  S}  €  B  (X)  , 


where  ny  -  (u^  , . . . )  ,  the  common  zeros  of  the  polynomials 
1  s 

nv€V(x  *  v  +  V  '  V  f  V(X)  '  are  the  Points  -  uy  ,  Y  6  8^  (X) .  Hence 
one  expects  t,  at  D^(x)  is  spanned  by  functions  of  the  form 

exp{-  uy  •  x}p(x),  where  p(x)  is  some  polynomial.  More  precisely,  let 
us  call  any  two  elements  Y  ,  Y'  £  equivalent  if  and  only  if 

UY  -  UY'  and  let  '  i  =  1 - - 'm  *  denote  the  corresponding  equiva¬ 

lence  classes  with  representers  V.  £  E.  ,  j  =  i,...,m  .  Moreover, 

define  for  any  Y  6  8  (x) 

s 

XY=  (z  €  X  :  z  .  uy  =  uz;  t 

i.e.  Y  c  xy  so  that  <Xy>  =  1RS  .  it  is  pointed  out  in  Dahmen  and 
Micchelli  (1987)  and  Ben-Artzi  and  Ron  (1987)  that  when  p 

*3  9  i  9 

i  =  1 , . . . ,  ,  is  a  basis  of  DQ(Xy  )  then  the  collection  of 

functions  3 


{exp(-  u 


'  x)Pi  4  (*)  :  i  =  1 . 1.  ,  j  =  1, 

j  J 


forms  a  basis  for  D^(X)  . 

The  spaces  DQ(x)  ,  D^(X)  play  a  fundamental  role  for  the  theory  of 
box  or  cube  splines  and  exponential  cube  splines  (cf.  Dahmen  and 
Micchelli  (1987),  de  Boor  and  Hollig  (1982/83))  leading  among  other 
things  to  interesting  implications  concerning  related  combinatorial 
and  algebraic  problems  (see  Dahmen  and  Micchelli  (1988),  (1987)).  More¬ 
over,  certain  interpolation  problems  were  recently  shown  by  Dvn  and 
Ron  (1988a),  (1988b)  to  have  unique  solutions  in  these  spaces. 

To  make  any  practical  use  of  these  results  requires  the  determination  of 
appropriate  bases  for  these  spaces.  In  view  of  (14)  this  reduces  to 
constructing  for  any  X  a  basis  for  DQ(X).  since  we  will  be  mainly 
concerned  with  this  space  we  will  henceforth  drop  the  sub- 


script  0.  It  was  shown  by  Dahmen  and  Micchelli  (1983a)  that  D(X) 
contains  only  polynomials.  In  fact,  one  has  the  inclusion 

nd(X)*S)  -  D(X>  ^"n-s®^'  (15)' 

where  d(X)  +  1  =  min{|v|  :  V  €  V  (X)  }  and  n^CIRS)  denotes  the  space  of 
all  real  polynomials  of  total  degree  at  most  k  onlRS.  Although  the 
dimension  of  D(X)  is  known  precisely,  its  structure  is  generally  rather 
complicated.  For  instance,  when  s  =  2  ,  X =  { ( 1 ,0) ,  ( 1 ,0) , (0 , 1 ) , ( 1 , 1 ) } ,  ' 

one  easily  verifies  that  {l,x,y,x  ,y  -  2xy}  is  a  basis  for  D(X)  in  ! 
this  case.  However,  in  general,  when  more  directions  are  involved  or 
when  dealing  with  more  than  two  variables  no  general  recipe  for  con¬ 
structing  a  basis  seems  to  be  known.  Exploiting  the  intimate  connec¬ 
tion  between  the  space  D(X)  and  certain  spline  spaces  a  general 
strategy  for  constructing  a  basis  for  D(X)  will  be  derived  in  the 
following  section. 

3.  A  basis  for  D(X) 

It  was  pointed  out  by  Dahmen  and  Micchelli  (1983a) and  deBoor  and  HSllig 
(1982/83)  that  when  X  C2S\{0)  the  space  D(X)  is  spanned  by  the  trans¬ 
lates  of  the  cube  spline  C(*|x).  If ,  in  addition, X  satisfies  (2)  then 
C  ( • | X)  is  related  to  the  truncated  power  by 

C ( • | X)  =  VXT(* |X)  , 

where  V  f(«)  =  f(»)  -  f(«  -  y)  and  for  v  £  V,  V  f  =  7  (V  .  ,f).  Hence 
y  v  v  vMv) 

appropriate  pieces  of  multivariate  truncated  powers  should  be  expected 
to  form  a  basis  of  D(X) .  It  will  be  shown  next  that  this  is  indeed 
the  case  even  for  arbitrary  X  c®sn{0}. 

Note  first  that  replacing  x^  in  X  by  cx^  for  any  c  £  1R  will 
leave  D(X)  unchanged.  Thus, assigning  appropriate  signs  to  the  elements 
of  X  ,  we  may  assume  without  loss  of  generality  that  X  satisfies  (2)  so 
that  T { • | X )  is  well  defined.  Moreover,  upon  rescaling  the  x^  if 


necessary  we  may  assume  throughout  the  following  that 


<xL>  =  <x^>  implies  xL  =  x^  . 


(16) 


li  s 

When  |  X I  =  n  =  s  there  is  nothing  to  prove  since  D  (X)  =  II  (JR  ) 

This  suggests  proceeding  by  induction  on  |x|.  Suppose  that  for  some  ' 

X  ,  |  X  |  =  n  >  s  one  has  already  determined  a  basis  for  D(X).  Further- 

more ,  suppose  that  4 


X-  =  X  U  {y} 

also  satisfies  (2) .  Then,  defining 

A(x|y)  =  (Y  €  6  (X)  :  ty}  U  Y  €  8  (X1) }  , 

S-l  s 


^  9 


one  has 


|Bs(X')|  =  1 A (x 1 y) |  +  |8s(X)|.  (17) 

Thus  it  remains  to  construct  N:=(A(x|y)l  additional  linearly  indepen¬ 
dent  elements  in  the  quotient  space  D(X')/D(X). 

The  construction  of  these  additional  polynomials  will  be 
based  on  an  appropriate  ordering  for  A(XQ |y),  where  XQ  c  x  is  a  maximal 
subset  of  pairwise  distinct  elements  in  X . 

To  this  end,  S  =  {s^  :  j  =  will  denote  a  fixed  maximal 

sequence  in  A(XQ|y)  such  that  the  cones  C..  =  <S..  U  {y}>  +  satisfy 


S'Wi * 0 


(18) 


Without  loss  of  generality  one  can  assume  that  no  element  of  S  can  be  written' 
as  the  union  of  essentially  disjoint  cones  of  the  form  <Y U  {y }>+,Y £  A (Xq |y) . 
In  fact,  if  such  a  cone  C  belongs  to  S  at  least  one  of  the  cones  C'  con¬ 
tained  in  C  cannot  belong  to  S.  Swapping  the  corresponding  elements 
Y  ,  Y'  of  A (XQ | y )  would  preserve  (18)  for  a  sequence  of  at  least  the  same 
length  L  • 


The  following  information  about  the  elements  of  A(XQjy)  which  are 
not  contained  in  5  will  be  useful. 

Lemma  1  For  every  Y  £  AfX^lyJ'sS  there  exists  a  subset  H  of  S  such  that 
<Y>+  c:  U{<S>+  :  S  €  H}  , 
i.e.  <Y>  =  <S>  ,  S  €  H  . 


Proof  Let  j  <  L  be  the  smallest  index  such  that 


C  =  <{y}  U  Y>  <=  u.  .C. 

+  -  K]  i 

i.e. 


(19) 


on  .C.  ^  0  - 
Kj  i 


(20) 


In  particular,  this  means  that 


K  =  <Y>+sUi<jCi  A  0 


and  K  c  c .  .  If  K  intersects  the  interior  of  C.  one  easily  concludes 
J  J 

that 


CjV((L'i<jCi)  U  C)  *  *  ' 


In  view  of  (20)  this  implies  that  inserting  Y  between  Sj_j  and  S.  still 

provides  a  sequence  satisfying  (18)  and  thus  contradicting  the  maxima- 

lity  of  S.  Hence  Kcr<s>  which  means  <s_,>=<Y>.  Let  vfS  ^Y.  Clearly 
~  J  +  j  D  1 

<Y>+  E  U{<{v}  U  ()Mz})>+  :  z  £  Y}  .  Since  every  <V>+  ,  V  £  A (XQ  | y)  ,  is 

covered  by  (s  -  1) -cones  <S>+  ,  S  £  S,  the  assertion  follows. 

An  immediate  consequence  of  Lemma  1  is  that  S  =  A(XQ|y)  if  the 

vectors  in  XQ  are  in  general  position.  In  particular,  this  is  the  case 

for  any  X  c®2  satisfying  (2). 

Now  let  6^  denote  the  (possibly  empty)  set  of  all  YfAfx^lyjNS  such 


that  the  highest  index  j  of  a  covering  H  for  <Y>+  given  by  Lemma  1  is 
as  small  as  possible.  Fixing  any  ordering  for  G_.  and  inserting  G  bet¬ 
ween  S_.  and  Sj  +  1  establishes  an  ordering  for  all  of  A(Xgjy)  which  will 
be  denoted  throughout  the  sequel  by  A(Xq|y)  =  {Y^  :  j  =  1 , . . . ,M} .  The 
corresponding  cones  <Y_.  U  {y}>  +  will  again  be  denoted  by  C  . 

The  role  of  AlX^iy)  with  respect  to  the  full  set  A(x|y)  becomes 
clear  when  considering  the  equivalence  relation 


Y  ~  V  iff  <Y  l'  {y}>+  =  <V  U  {y}>+  ,  Y  ,  V  E  A(x|y)  . 


In  fact,  recalling  (16),  one  easily  verifies  that  A(X^|y)  is  a  set  of 
representers  for  the  corresponding  equivalence  classes  E .  of 


Y  £  A  (Xq  [ y )  .  Note  that  for  Y^  =  {x'*' ' 1 , . . .  ,xx the  cardinalities 
£  =  |E  |  are  given  by 


i  ,s-l 


H .  — m,  . * .  m .  .  , 

1  r,l  l , s— 1 


(21) 


i  k  3 — i 

where  x  '  occurs  m.  times  in  X  .  For  any  1=  (i  ,...,i  )  E  tl  let 

1 » K  1  S—  1  + 

_  /  -r  \  r  1  f  1  i  /  1  i  ,  S-l  1  /  S”l  i 

Y^(X)  —  i x  t • • • t x  f • • • f x  t  • • • r x 

i  k 

where  x  '  occurs  exactly  i  times  in  Y^I)  .  Specifically, 

Y^  ( ( 1 , . . . ,  1 ) )  =  Y^, while  will  be  used  as  a  shorthand  notation  for 
Y .  ( (m .  , . . .  ,ra  ) )  .  Let  £(i)<i  be  the  largest  integer  such  that 

i  if l  lfSi 

Y*u)e  S-Let  Ai=^(i)>+"uj<ii(i)<Yj>+  and  define  for  Vs 


Hi  =  £  j  <  i  *  <Yj>  =  <Yi>,<Y>+  Hi.  /  0N$i  , 


(22) 


while  H.  =  0  when  Y.  £  S.  Furthermore,  let 
i  i 


Ji  =  {I =  (ij , . . . ,is_1)  :  ik  £  {0, . . . ,mi  k~l }  ,  k  =  1 , . . . ,s-l } ,  (23) 


so  that  ! (  =  SL  =  ; | .  Finally,  defining/ for  I  £  J,» 


VCI) . =  YtII)  U  H±  , 


(24) 


one  is  ready  to  construct  additional  linearly  independent  polynomials 
in  D(X')M)(X)  .  Writing  briefly  D  f  =  (nv£vDv>f  the  polynomials 

P  (x)  ,  I  €  J.  ,  i  =  1 _ _ ,M  ,  are  defined  to  be  extensions  of 

i  1 1  i 

certain  polynomial  pieces  of  truncated  powers,  namely 


Pi,i(x)=Dv(i)T(x  X<) 


x€  ^(i^Mi^  nCi' 


(25) 


The  main  result  of  the  paper  may  be  stated  as  follows. 


Theorem  2  Let  X  ,  X'  =  X  U  {y}  satisfy  (2)  and  let  P  be  defined 

— —  i.  —  1/1 

by  (25).  Then  for  any  basis  8  of  D(X)  the  set 


8  U  {P.  _  :  I  €  J  ,  i  =  1,...,M} 

1  f  ±  X 

is  a  basis  for  D(X'). 

*> 


Proof  Note  that  in  view  of  (21),  (23)  there  are  exactly 

M  M 

r  IjJ  =  r  ! Ei |  =  | a (x | y )  |  =  n 
i=l  i=l 


functions  P.  .  In  view  of  (12)  and  (17)  it  remains  to  show  that  the 

i  1 1 

P,  represent  linearly  independent  elements  in  D(X')/D(X). 
i  1 1 

To  this  end ,  recall  from  (9)  that  for  any  vczx1, 

DyT ( 0 j  X ' )  =  T ( • |X'^V)  .  (26) 

Since  D (W)  c  D(X)  for  W  c  x,(10)  readily  confirms  that 


P  €  D (X ' )  ,  I  €  J.  ,  i  =  1,...,M  . 
i  f  i  1 

Next  observe  that  for  x  €  (°£  (i)'sUj<i  (i) C j )  n  Ci  ' 


(27)  _ 


i 


4) 


■W>pi,i“  '  ■W>Dv<i)T<*|x') 

l  l 


=  T  {x  j  {y}  U  (X  fl  <Yi>)^V(I)  ) 


By  definition  (24)  of  V(I),the  set  (X  f)  <Y.>)M/(I)  still  contains  some 

J  l 

element  of  A(xjy)  while  also  (cf.  (3)) 


Cin  (CZ(i)'sUj<£(i)Cj>  -<{y}  U  "xn<V)SV(I»>+ 


=  supp  T(*  |  {y}  U  (  (Xfl  <Y  >)>V(I)  )  )  . 


Hence  P.  does  not  vanish  identically.  But  since  Xvy.>£  V(X)  , 

i  .  ^  1  j  X  X 

this  means  that 


P  €  D (X)  ,  I  €  J.  ,  i  =  . 

1  /  X  X 


So,  in  view  of  (27)  ,  (28) ,  it  remains  to  confirm  the  linear  indepen¬ 
dence  of  the  P.  in  D(X')/D(X),  To  this  end,  the  following  observation 
is  useful. 

Lemma  2  For  i  =  1 , . . . ,M  ,  M  =  |A(XQ|y)|  ,  let 


Wi  =  {y }  U  Vfm1)  , 


where  m  =  (m  ,...,m.  )  .  Then  the  functions  D  P.  ,  I  6  J.  , 

i rS-i  x  .  i r x  x 

are  linearly  independent. 


Proof  Suppose 


riexciIWpi,i(x)  =  0  '  x  G*  * 

i  i 

By  definition  (25)  ,  (26)  and  (9)  this  implies 


IX€JiCIT<x|wrV(I))  =  °  '  X  €  (C£(i)VUj<£(i)Cj)  0  Ci 


By  (24)  one  has 


{W>V(I)  :  I  €  J.  }  =  { iy  >  U  Y.  U  Y.  (I  •  )  :  I*  £  J .  }  . 

l  ill  l 


Hence  the  summands  T ( *  j VT^V ( X) )  ,  I  £  J  ,  range  over  all  different 
products  of  the  form  (7) .  Since  these  functions  are  obviously  linearly 
independent  one  concludes  c^  =  0  ,  I  £  .  This  completes  the  proof 

of  Lemma  2 . 

Next  suppose  that  for  a  given  basis  {Q . : j =  1 , . . . , i B  (X) I }  of  D(X) 

3  s' 

|8  (X)  | 

Zl£J.Ci,IPi,I (X>  +  Zj  =  l  a323 (X)  =  °  '  X  €  ^  * 

Since  X,v^?^€V(X)  one  also  has  for  any  k  <  M, 


^=1  Zl£j.Ci,IDX>vw  Pi,I(x)  =  °  '  X  *** 


(29) 


Lemma  3  For  any  I  £  J .  ,  i  >  k, one  has 


Vv*  pi,i(x)  =  0  '  X  . 

k 


(30) 


Proof  Note  that  for  fixed  i  and  x  £  (C...,V4J.  .  ...C.)  (I  C.  , 
-  £(i)  j<l (i )  3  l 


[Wpi,i(x>  "  Dv<l)T<xix'MX,''V> 

k 


(31) 


=  Dv(i)T(x|V- 


Let  k<  i  and  suppose  first  Y  €S  ,  i.e.  l[ i) 


=  i.  Note  that  in  this  case 


C  V<W  >  1-  0  .  (32) 

1  jc  ■» 

In  fact,  (32)  readily  follows  from  (18)  and  (22)  when  <Y^>  /  <Y^>  or 

when  Y  £S.  So  suppose  <Y.>  =  <Y  >  and  Y  (?  S.  Let  R  denote  an  ( s— 2 )  — 
k  i  k  k 

dimensional  supporting  hyperplane  of  A^  that  separates  A^  and  A^.  By 


•  • 


i 


(22)  those  Y  that  contribute  to  H,  must  be  on  the  same  side  of  R  as 
m  k 

A,  ,  again  confirming  (32).  Thus,  when  Y.  f  S  (30)  follows  from  (3),  (31) 

K  i 

and  (32).  The  same  reasoning  applies  if  Y  g  S  but  k<  Jt(i).  So  assume 
next  that  i(i)<k<i.  Hence  <Y.>  =  <Y  >.  From  (22),  (24)  one  concludes 

X 

that  dinKW.-'V ( I ) >  <  s  so  that  (30)  follows  again  from  (3),  (31)  and 
(26).  This  completes  the  proof  of  Lemma  3. 

Choosing  now  successively  k  =  1,2,...,M  in  (29)  and  invoking 
Lemma  2  at  each  step  shows  that  c.  =  0  ,  I  f  J.  ,i=l,...,M.  This 

1  /  X  1 

proves  the  linear  independence  of  the  polynomials  P.  in  D(X')/D(X) 

i  r  i 

which,  in  view  of  (12),  (17)  finishes  the  proof  of  Theorem  2. 

As  pointed  out  in  Section  2  (cf.  (14))  being  able  to  construct  a 
basis  for  spaces  of  the  type  D(X)  immediately  allows  to  exhibit  a  basis 
of  D^(X)  for  any  (i  €  <Cn  .  Alternatively,  using  the  operators  p.  +  D^j 
instead  of  D  j  ,  and  (8),  a  basis  for  D  (X)  could  be  constructed 

X  P 

directly  from  the  pieces  of  T  (*|x)  in  the  same  way  as  shown  above. 

4 .  An  example 

In  this  section  possibilities  of  evaluating  and  representing  the  above 
basis  functions  will  briefly  be  discussed  and  illustrated  by  an  example. 

One  should  note  first  that  under  additional  assumptions  on  X  the 
above  construction  may  simplify  significantly.  For  instance,  when  X 
has  the  form  X  =  {y1 , . . . ,yl , . . . ,yS , . . . ,yS }  where  Y  =  {y1 , . . . ,yS}  spans  TRS, 
one  has  [ A (X  |y) |  =  1.  In  this  case  (25)  produces  in  view  of  the 
representation  (7)  the  expected  tensor  product  basis.  When  the  elements 
of  X  are  in  general  position  one  has  A(x|y)  =  A(XQ|y)  =  S  and  the  diffe¬ 
rent  pieces  of  T(« |v)  ,  V  c  X  ,  form  a  basis  of  D(X)  .  In  this  case  the 
following  explicit  representation  for  T(*|x)  was  established  by  Dahmen 
and  Micchelli  (1986): 

T(x|x)  =  Tn-s)'i  1  aJdet  y|  1(xY  •  x)n_Sx  (x)  , 

(n  5)1  Y€8  (X)  Y  <Y>+ 

s 

where,  for  Y  €  8  (X)  , 
s 


(33) 


xY  •  y  =  1  ,  y  6  Y 

and 

a  =  IT  (1  +  xY  •  z)  1  . 

Y  zgY 

A  similar  representation  can  be  given  for  T^(*  iX)  when  p_.  i  0  ,  j  =  1 , ...  ,n 
(cf.  Dahmen  and  Micchelli  (1981)). 

When  X  is  in  general  position  another  explicit  basis  was  construe-  * 
ted  by  Dahmen  and  Micchelli  (1985)  without  using  truncated  powers.  i 

In  general,  one  has  to  use  the  recurrence  relation  (cf.  Dahmen  3 

(1980) ) 

T  (x  I X)  =  — Xn  .  \  ,T(xjxMx^})  (34) 

n-s  3=1  3 

which  holds  whenever 

x  =  .  (35) 

A  general  strategy  would  be  to  select  first  a  possibly  large  subset  X1 
of  X  for  which  a  basis  is  easily  available  either  via  (7)  or  (33) .  The 
representations  for  the  extensions  of  type  (25)  could  then  be  obtained 
by  applying  the  recursion  (34)  .  The  freedom  in  choosing  the  represen¬ 
tations  (35)  should  then  be  exploited  in  such  a  way  that  possibly  many 
of  the  previously  calculated  representations  occur  during  the  recursion. 
This  is  in  fact  strongly  favoured  by  the  fact  that  (25)  always  involves 
truncated  powers  for  subsets  of  X  . 

The  following  bivariate  example  illustrates  this  strategy.  As 
pointed  out  before ,  after  assigning  appropriate  signs  to  the  elements 
of  X  ,  so  that  0  ?  [x]  ,  Xg  will  be  always  in  general  position.  Hence 
S  =  A(xQ|y)  and  =  0  (cf.  (22))  for  all  i  ,  which  simplifies,  of 
course,  the  construction. 

Let  X  =  {x1,...^6}  cm2  where  x1  =  x2  =  (1,0)  ,  x3  =  x4  =  (0,1)  , 

5  6 

x  =  (1,1)  ,  x  =(-1,1).  In  order  to  construct  a  basis  for  D(X)  one 
can  follow  the  lines  of  the  previous  section  extending  step  by  step 
bases  of  appropriate  subspaces  of  D(X)  .  For  instance, consider 


i 

11  4 

X  =  {x  , _ ,x  }  .  As  pointed  out  before  the  representation  (7)  already 

indicates  the  tensor  product  structure  of  D(xS  and  one  easily  verifies 
that 


D(X1)  =  span! 1 ,x1 ,x2 }  .  (36) 

2  1  5 

Setting  X  =  X  U  (x  },  one  has 

ACX^x5)  =  { {x1 } , tx2 } , {x3 } , {x4} }  . 

1  3 

One  can  take  Y  =  {x  } ,  Y.  =  {x  }  as  representers  of  the  corresponding 
*  l  5 

equivalence  classes  in  A(X  x  )»with  J  =  J 2  =  {  (0) , (1) }  .  According 
to  (25)  one  has 

P1  (0)  (x)  =  T(xix2)  '  Pi  (i)  (x)  =  T(xix2Mx1}) 
for  x  £  Cj  =  <{x1,x5}>+  and 

P2  (0)  (X)  =  T(jc|x2)  ,  P2  (1)(x)  =  T(x|x2^-{x3}) 

for  x  €  C2  =  <{x3.x5}>+  .  By  (34)  one  obtains,  for  x  £  Cj, 

2  1  2  1  1 
T  (x  |  X  )  =  -(  (Xj  -  x2)T(X|X  Mx  })  f  x2T(x|x  )  )  . 

Using  (34)  again  yields 

T  (x  |  X^tx1 })  =  -j((x1-x2)T(xjx3,x4,x5)  +  x2T  (x  |  x2  ,x3  ,x4) )  , 

which  for  x  €  C1  reduces  to 

X2  ,  2  3  4,  2  . 

—  T(x|x  , x  ,x  )  =  x2/2  . 

Likewise,  T(x  |  X1)  =  XjX^so  that 


«> 


♦ 


•  • 


i 


(37) 


P1  ,  (0)  (X)  2  (X1X2  3  X2) 


Similarly,  for  x  €  C^, 


1,,  1 


TCxixMx1  })  =  j(  (x1  -  x2)T(x|x3,x4,x5)  +  x7T(x;x^  ,xJ  ,xq) )  =  -j-  , 


2  3  4, 


so  tha. 


P1,(1)(X)  =  X2/2 


(38) 


By  symmetry  one  only  has  to  interchange  x^  and  x to  obtain 

r,  ,  ,  1  ,  2  13. 

P2  ,  CO)  x  =  2(X2X1  -  TV  ' 

P2,(1)(X)  =  1  A  ’ 


(39) 


so  that  by  Theorem  2  D(X  )  is  spanned  by  the  polynomials 


1  '  X1  '  x2  *  x!x2  '  xi  '  x2  '  xix2  “  3  X2  '  X2X1  '  I  X1 


Finally  let 


X°  =  X2  U  {x6}  , 


so  that 

AUV)  =  {{x1}  :  i  = 

Setting  Y1  =  {x3},  Y2  =  {x5},  Y3  =  {x1},  Jj =  { (0) ,  (1) }  =  , 

J2  =  {  (0) }  ,  one  has  Cj  =  <{x3,x6}>+  ,  C2  =  <{x5,x6}>+  ,  C3  =  <{x* ,x6}>+  . 
Cj# clearly  satisfy  (18).  Hence,  according  to  (25)  one  obtains 

P1(0)(x)  =  T(x|x)  ,  Pul)(x)  =  T(x|x^{x3}) 

for  x  €  Cj  , 


I 


*> 


■§> 


and 


2,(0) 


3,(0) 


(x)  =  T (x  X)  ,  x  €  C2^ci 


(x)  =  T(xjX)  ,  P,  ,,,(x)  =  T(xix^{x!}) 


3,(1) 


for  x  €  C3"C2 


Repeated  application  of  (34)  yields 


pl, <0) 


P2. (0) <xl 


P3,<0)<x> 


P3,U)<Xl 


(x 1  +  x2)4/48  ,  P1  (1) (x)  =  (xt  +  x2)3/12  , 

12  >21  2 ,  .  1  ,  ,,213, 

48  X2(X1  +  X2>  +  24  X1X2(X1  +  X2}  +Ie<X2  'V  (X2X1  '  iV' 


1  ,  ,31.314, 

24(X1  *  X2)X2  +  8(X1X2  -  3  V' 


1  3 

6  X2  • 


One  easily  verifies  (cf.  (14))  that 


dim  D (X)  =  ! B  (X) |  =  13  . 


According  to  the  preceding  calculations  a  basis  for  D(X)  is  then  given 
by  the  following  thirteen  polynomials 

,  2  2  2  1  3 

1  '  xi  ’  x2  ,  V2  '  x!  '  X2  '  X1X2  "  I  X2  ' 

2  1  3  3  ,  ,3  ,  ,4 

X2X1  -  3  X1'X2'  (X1  +  X2  '  (X1  +  X2  ' 


12,  , 2  1  2,  ,  ^  1  , 

48  X2(X1  +  X2)  +  24  X1X2(X1  +  X2>  +  Ie(x2 


1  ,  ,31,314, 

24  X1  +  X2)X2  +  8(X1X2  “  3  X2)  ’ 


,  ,  2  13, 

V  (x2xl  "  3  xl}  ' 
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MONOTONE  PIECEWISE  CUBIC  DATA  FITTING 


F.  N.  FRITSCH 

Computing  and  Mathematics  Research  Division 
Lawrence  Livermore  National  Laboratory 


Abstract  This  paper  describes  PCHLS,  an  algorithm  for  least  squares  fitting  of 
a  monotone  piecewise  cubic  function  to  data.  It  extends  the  piecewise  cubic  Her- 
mite  interpolation  package  PCHIP  to  situations  in  which  the  data  are  noisy  or  are 
adequately  represented  by  far  fewer  cubic  pieces  than  data  points. 

Key  words:  Data  fitting,  Least  squares,  Data  reduction,  Piecewise  cubic  approxi¬ 
mation,  Monotonicity  preserving  approximation.  Shape  preserving  approximation 


1.  Introduction 

Necessary  and  sufficient  conditions  for  a  piecewise  cubic  function  to  be  monotonic 
were  published  in  [1].  PCHIP,  a  complete  package  for  interpolation  with  piecewise 
cubic  functions  and  evaluating  the  results  has  been  in  use  for  many  years  [4].  PCHLS 
is  a  new  algorithm  to  do  least  squares  fitting  of  a  monotone  piecewise  cubic  function 
to  given  univariate  data  (z*,y*),  k=l,. . .  ,m.  Its  development  was  motivated  by 
user-expressed  needs  to  extend  PCHIP  to  noisy  data,  for  which  interpolation  is  not 
appropriate,  or  to  cases  in  which  it  is  clear  that  an  adequate  approximation  could 
be  achieved  with  far  fewer  cubic  pieces  than  data  points  (data  reduction). 

In  Section  2  is  given  the  mathematical  statement  of  the  problem  solved  by 
PCHLS,  and  algorithmic  details  are  described  in  Section  3.  Several  examples  to 
illustrate  the  results  of  the  algorithm  appear  in  Section  4.  Directions  for  further 
work  are  indicated  in  Section  5. 


2.  The  Mathematical  Problem 


Let  [a,  6]  be  aa  interval  containing  all  of  the  data  points  x*  and  define  breakpoints 
t,  so  that 


®  —  fl  <  tj  <  . . .  <  tnjea  <  tn  —  6, 


where  nsejji  =  n  —  1.  A  piecewise  cubic  Hermite  (PCH)  function  with  breakpoints 
ft  has  the  form 


A*)  =  +  **?(*)], 


where  d.= /'(<.)  and  fff,  fiTJ*  are  the  cubic  Hermite  basis  functions  defined 

by 


where  we  have  used  the  standard  Kronecker  delta  notation. 


2.1  Least  squares  equations 

The  least  squares  equations  for  the  unknown  PCH  parameters  fi,dj  are  obtained  by 
evaluating  (2)  at  each  of  the  data  points: 


£  [f*  *<(**)  +  *  #.'(**)]  =  »*»  *  =  1 . m. 


We  note  that  if  the  2n  unknowns  are  arranged  in  pairs  /i,di,/2,dj,. . .  ,/„,d„  the 
A:-th  row  of  the  m  x  2n  least  squares  matrix  will  be 


*?(**)  Hfok)  H*(xk)  ...  *'(**)  ^(x*)]  . 


Since  the  i-th  Hermite  basis  function  is  zero  unless  x*  €  the  matrix  will 

have  a  block  structure  with  (at  most)  four  nonzero  entries  in  any  row,  as  indicated 
in  Figure  1. 


2.2  Monotonicity  constraints 

To  the  least  squares  equations  (5)  must  be  added  the  monotonicity  constraints 


0  <»i-idi<  3«,_i  j »  =  2,...,n; 
*«  -  *«— i 


0  <  Sidi  «=  l,...,n-l. 

‘«+i  -  U 


•  « 


>  •—  data  points  in  (t\,t2) 


X  X  X  X 
X  X  X  X 

X  X  X  X 

X  X  X  X 
X  X  X  X 


X  X  X  X 

X  X  X  X 
X  X  X  X 

X  X  X  X 


•—  data  points  in  (t2,tj) 

. data  points  in  (t3,t4) 


Figure  1.  Structure  of  least  square*  matrix 

Here  s,  will  either  be  +1  or  -1  according  as  the  approximation  is  to  be  monotone 
increasing  or  decreasing  in  (t„t1+i).  These  sufficient  conditions  for  montotonicity 
come  from  requiring  that  the  derivatives  lie  in  the  square  [0,3]  X  [0,3]  in  the  mono- 
tonicity  region  of  [l].  Conditions  (7)  and  (8)  constitute  4(n  -  1)  linear  inequalities1 
among  the  unknowns  Each  involves  either  one  or  three  of  the  unknowns  and 

has  a  block  structure  similar  to  that  of  the  least  squares  equations. 


•  4 


3.  The  Algorithm 


PCHLS  is  given  data  (z*,yfc)  and  weights  wk,  breakpoints  t,  satisfying  (1),  and  an 
array  of  constraint  flags  s*: 


Si  = 


< 


+1 

-1 

0 


for  monotone  increasing  in  (t,-,f,+i), 
for  monotone  decreasing  in  (f;,  <«+i), 
for  no  constraint  in  (f,-,  t,+i). 


(9) 


PCHLS  is  to  return  PCH  coefficients  (/<,d,),  t=l,...,n  in  (2)  which  minimize  the 
sum  of  squares  of  weighted  residuals 


£  («>*[/(**)  -  y*])3 


*=i 


(10) 


subject  to  the  constraints  (7)  and  (8). 


1  These  will  not  ill  be  independent.  In  the  common  case  in  which  the  approximation  is  to  be 
increasing  throughout  [*,&],  for  example,  there  will  be  only  n  +  2(»  -  1)  constraints. 


l 


The  algorithm  operates  as  follows. 


Step  1. 

Sort  the  data  so  that  xi  <  x-i  <  . . .  <  xm.  (This  is  actually  done  externally  to 
PCHLS,  to  avoid  extra  work  in  case  the  data  arrives  ordered.) 

Step  2. 

Evaluate  the  cubic  Hermite  basis  functions  at  the  data  points  and  set  up  the  least 
squares  matrix.  Weights  are  included  by  multiplying  the  k- th  row  (6)  and  the  right- 
hand  side  yk  by  tu* . 

Step  3. 

Set  up  the  monotonicity  constraints  (7)  and  (8).  No  constraints  are  generated  for 
intervals  in  which  s,=0.  Redundant  sign  constraints  are  eliminated  when  a,_i=Sj. 

Step  4. 

Solve  this  linearly  constrained  linear  least  squares  problem  via  SLATEC  subroutine 
LSI,  which  is  based  on  the  algorithm  described  in  [5]. 

Step  5. 

Set  “small”  d, -values  to  zero.  (This  postprocessing  step  is  necessary  because  the 
current  version  of  LSI  often  returns  values  the  size  of  the  unit  roundoff,  possibly 
with  the  wrong  sign,  when  one  of  the  sign  constraints  is  binding.) 

Since  the  hard  part  of  obtaining  a  satisfactory  fit  is  generally  the  choice  of 
breakpoints  and  constraints,  an  interactive  driver  has  been  built  on  top  of  PCHLS.  It 
reads  data  and  weights,  translating  absent  weights  to  Wk= 1  •  After  the  user  sets  nseg, 
it  generates  either  uniform  or  equidistributed2  breakpoints  and  automatically  selects 
the  Sj  by  examining  the  data.  It  performs  Step  1,  above,  calls  PCHLS,  and  does 
Step  5.  It  plots  the  approximation,  with  data  points  and  breakpoints  superimposed, 
after  each  fit.  The  driver  allows  the  user  to  move  or  delete  breakpoints,  change  sign 
constraints,  or  change  the  number  of  segments. 


4.  Examples 

This  section  contains  some  plots  generated  by  the  PCHLS  driver,  to  illustrate  the 
capabilities  of  the  algorithm.  All  fits  were  unweighted. 

The  first  example  is  the  titanium  data  from  [3].  This  is  uniformly  spaced  and 
appears  to  contain  a  moderate  amount  of  noise.  Figure  2(a)  uses  naeg= 7  with  all 
default  settings  and  is  clearly  a  poor  fit.  Adjusting  the  positions  of  breakpoints 
3-5  and  setting  a,=+l  in  the  first  four  intervals,  -1  in  the  last  three,  yields  2(b). 
Changing  *«  to  zero  improves  the  RMS  error  from  .0026  to  .0018  (maximum  relative 
error  from  8%  to  5%).  (See  Figure  3(a).)  Deleting  the  second  breakpoint  has 

2  Same  n amber  of  data  point*  in  each  aegment. 


■  |.I2«-4I.  rests  ■  ratirr  ■ 


■eaei  •  rnia  ■  7  <}«-•!,  raserr  ■  M4*~9t 


Figure  2.  Two  fits  to  titanium  data,  (a)  The  result  with  all  default  settings,  (b)  The  resnlt  after 
adjusting  middle  breakpoints  and  constraints. 

virtually  no  effect  on  the  result,  as  indicated  in  Figure  3(b).  In  Figure  4  this  curve  is 
compared  with  the  PCHIP  interpolant.  The  latter  has  2m=98  parameters,  whereas 
the  PCHLS  fit  has  only  2n=14,  so  we  have  smoothed  out  the  noise  and  achieved  a 
data  reduction  factor  of  7. 

The  second  example  illustrates  the  use  of  equidistributed  breakpoints.  The 
assumption  here  is  that  if  the  data  are  not  uniformly  spaced,  then  the  sample  rate 
was  dictated  by  the  behavior  of  the  phenomonon  under  study.  This  is  certainly  the 
case  with  the  potentiometric  titration  data  from  [6].  In  figure  5(a)  is  the  PCHLS 
result  with  seven  equidistributed  breakpoints.  This  has  2n=14  parameters  and  an 
RMS  error  of  18  (maximum  relative  error  1%).  It  looks  much  the  same  as  the  PCHIP 
interpolant  in  Figure  5(b),  which  has  2m=42  parameters. 


5.  Further  developments 

The  most  obvious  improvement  needed  would  be  a  constrained  linear  least  squares 
solver  that  could  take  advantage  of  the  Bparseness  of  the  problem.  The  least  squares 
matrix  is  m  x  2n  but  has  only  4m  nonzero  elements.  If  neon  <  4(n  -  1)  is  the 
actual  number  of  constraints,  there  will  be  fewer  than  3 neon  nonzero  elements  in 
the  neon  x  2n  constraint  matrix.  While  the  wasted  space  is  tolerable  in  most  of 
the  problems  on  which  PCHLS  has  been  used  to  date,  it  will  be  prohibitive  if  the 
algorithm  is  to  be  extended  to  bivariate  data. 

The  present  automatic  constraint  setting  algorithm  is  very  primitive.  It  might 


(a)  (b) 


Figure  3.  Two  minor  adjustments,  (a)  Removed  constraint  in  fourth  segment,  (b)  Deleted  first 
interior  breakpoint. 
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(a)  (b) 

Figure  A.  Comparison  of  the  latter  curve  (a)  with  the  PCHIP  interpolant  (b)  to  the  titanium  data. 


(a)  (b) 

Figure  S.  Two  approximations  to  potentiometric  titration  data,  (a)  The  PCHLS  fit  with  n=7  and 
eqnidistribnted  breakpoints,  (b)  The  PCHIP  interpolant. 

be  improved  by  doing  local  least  squares  fits  of  straight  lines  to  the  data  in  each 
breakpoint  interval. 

Because  the  most  tedious  part  of  the  fitting  process  for  a  user  is  adjusting 
the  locations  of  the  breakpoints,  work  on  automatic  knot  placement,  such  as  that 
reported  by  Peter  Harris  at  this  conference  [2],  might  be  a  valuable  addition  to  the 
PCHLS  driver. 

The  author  has  had  good  results  on  some  problems  by  adding  convexity  con¬ 
straints  to  the  monotonicity  constraints  provided  by  PCHLS.  This  modification 
comes  closer  to  providing  a  true  shape  preserving  data  fitting  algorithm. 

Another  modification  under  consideration  is  the  addition  of  a  term  to  the  ob¬ 
jective  function  to  reduce  the  size  of  the  second  derivative  jumps  at  the  data  points. 
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DIRECT  AND  CONVERSE  RESULTS  ON  SIMULTANEOUS 
APPROXIMATION  BY  THE  METHOD  OF  BERNSTEIN -DURRMEYER 
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Abstract  The  n  -th  Bernstein  -Durrmeyer  operator  Mfl  results  from  the  n  -th 

classical  Bernstein  operator  with  weights  pQ^  if  the  discrete  values  in  its 
definition  are  replaced  by  an  integral  over  the  weighted  function.  For  integrable 
functions  the  rate  of  simultaneous  approximation  will  be  related  to  the  Ditzian  - 
Totik  modulus  of  smoothness.  For  the  local  rate  of  simultaneous  approximation  a 
generalized  Voronovskaja  theorem  is  proved. 

Key  words:  Bernstein  -Durrmeyer  operators,  Direct  results,  Inverse  results,  Local 
direct  results,  Simultaneous  approximation. 


I.  Introduction 

The  n-th  Bernstein -Durrmeyer  operator  Mn,  n  €ll,  results  trom  the  n-th  classical 

Bernstein  operator  with  weights  pQ^(  x)-(E)xk(l-x)"'l,k€{« n  }, 

x  €  I  =  [0,1],  if  the  discrete  values  f(k )  in  its  definition  are  replaced  by  an  integral 
over  the  weighted  function.  More  precisely  MQ  assigns  to  a  function  f  €  L^(I)  the 
polynomial  MQf  of  degree  n  defined  by 

(Mnf)(x)  =  (n+1)  JoPnk(x)/pnk(t)f(t)dt,  x  6 1. 

These  operators  were  introduced  by  Durrmeyer  (1967)  and  first  studied  by 


’■i 


Derriennic  (1979).  For  the  rate  of  approximation  in  the  Lp -metric  globed  direct  and 
inverse  theorems  have  been  proved  by  Ditzian  and  Ivanov.  Their  paper  contains 
moreover  asoiution  of  the  so-called  nonoptimal  approximation  problem.  The 
global  saturation  problem  has  been  solved  by  Heilmann. 

In  this  paper  we  will  derive 

a)  global  direct  and  inverse  results  for  the  rate  of  weighted  simultaneous 

approximation  |jw(Mnf-f)^8^||p)  s  6  *o,  of  functions  f  €  L®(I) :  =  {g|g^€  Lp(I)}, 

1  <  p  <  oo (Theorems  3,  5,  6,  7), 

b)  a  generalized  Voronovskaja  theorem  for  the  local  rate  of  (nonweighted) 
simultaneous  approximation  (Mnf-f/s^(x),  f  6  L®(I)  (Theorem  8). 

If  f  €  Lp(I),  s  €  io,  1  <  p  <  oo,  n  >  s,  x  6  I,  then 

(Mnf)W(x)  =  U+lMMlY, Pn^it(x)^pn+.k+s(.)l<3>(t)dt,  (1) 

/  |  \2 

with  a(n,s)  =  ^TiWJ[+s7<  cf‘  Derriennic (1979). 

Occasionally  we  will  work  for  convenience  with  operators  g  given  for  h  6  Lp(I), 
1<P<00,  n>s,  x€lby 

(Mn  sK)(i)  =  (»+D«(", »)kS0P„.a,k(*)]pn+Sik+s(t)h(t)dt .  (2) 

Evidently  (Mnf)(s)  =  Mn  iff  €L*(I).  (3) 

Throughout  this  paper  Cj  will  denote  positive  constants  not  necessarily  the  same  at 
each  occurance. 

2.  Basic  properties 

For  every  n  6  i,  k  €  {0,...,  n},  x  €  I,  there  holds 


k=0P"kUI  =  1’  ^pnk(t)dt  =  ETT'  <4> 

|pnk(x)-x-pn.lk.i(x),  /* '  Pn+5,k+s<,)<11  =  (n+etnUs+'J)  ’  (6) 

p;kW  =  "(P„.l,k-lW-P11.lil(x)),  (6) 

^I*2pnk(:t)  =  (t'"l|Pnk,xl'  (7) 


(Here  and  in  the  following  we  put  formally  pn^(x)  —  0  whenever  k  <  Oor  k  >  n.) 


v 


Using(4)  and  the  Riesz-Thorin  theorem  (cf.  Bergh,  Lofstrdm,  Theorem  1.1.1)  it  is 
easily  proved  that 

HMn,shHp  *  CIMIp>  h  €  LpW’  1  <  P  <  *>  "  >  s,  (8: 

with  a  constant  C  that  is  independent  of  n  and  p. 

The  following  lemma  is  useful  in  connection  with  estimates  for  the  moments 
Mn8(t-x)m(x),m€llo. 

Lemma  1  Let  m  6  i0,  s  €  HQ,  n  >  s,  x  6  I  and 

=  (“+s+l)kfoPn-s,k(l)^»+>,k+s('),,-l)m‘"- 

With  y(x)  —  ^x(l-x)  we  have  the  recursion  formula 

Tn,s,oW=1'  '10: 

Tn,s,m+l^x^ 

=  HTiTSTFT[»<I)2(T;.S!m(:‘>+2n,-T»,s,m.l(I)»+('+m+1Kl^)T„,,  ,m<4 

m  €  M,  and  the  representations 


m  /_v2  m-i  ~ 

■  <u> 
m  /  \2  in  “i  ,v  i 

TmW«>  =(1'2l)if06i,2m+l<"M1^  "  ' 

where  the  8j  2m(n) an<^  ®}  2m+l^  316  *n<kp«ndent  of  x  and  for  fixed  m  uniformly 
bounded  in  n. 


2  m-i 


Proof  T  n(x)  and  T  ,(x)  can  be  calculated  directly.  The  recursion  formula 
IlySyU  n,S,l 

follows  by  an  argument  which  is  similar  to  Oerriennic  (1979),  Proposition  II.3. 
Then  the  representations  (11)  cam  easily  be  derived  from  (10)  by  induction. 

In  section  3  the  following  corollaury  will  be  needed. 


Corollary  2  For  every  m  €  HQ,  n  >  s,  n  >  3  and  x  €  I  we  have 

|Mn,(t-x)2m(x)|  <Cn-mMi)2+n-1)m, 

|M„  s(t  -x)2m+l(x)  |  <  Cn  ■m-1(  *<x)2+n  •1)m  . 
where  C  is  independent  of  n. 


I 


ll 


I 


Proof  For  x  €  [0,^]  U  (1  -  pi]  we  have  $x)“  <  Thus  by  (11) 


|M„3(t.l)2-n(x)|<^;});}"|S)|o|a.2m(n)l(^)m''-n-2i<c. 

For  x  €  [pl-^]  we  have  [n^x)~]  **  Thus  by(ll) 

I 1  i  (°+U?il,S) " "”^I|2m|0 1 W"> 

<Cn‘m^x)2m. 

The  second  estimate  of  ( 12)  is  proved  analogously. 


-2i  ,  r>-  -2m 


For  the  two  monomials  eg  and  e^  we  obtain  by  direct  computation 

(M„  ,e0Xi)  =  1H-OC-1)  ,  =  i+0(n_1)  (13) 

uniformly  for  x  €  I  and  n  -» ®. 

In  our  first  global  direct  theorem  the  rate  of  (non weighted)  simultaneous 

approximation  ||(Mnf-f)^||  will  be  estimated  using  the  second  order 
Ditzian-Totik  modulus  of  smoothness.  In  our  case  the  r-th  order  modulus  of 
smoothness  ( r  €  M)  is  given  by 

w^(f,t)  =  sup  ||Aj  f||  ,f€L  (I),  1  <  p  <  «,  ?(x)  =  Jx:(l-x),  where 
P  p  0<h<t  nv?  p  p 

Aj  f(x)  =  S  (-l)k(J)f(x+(2-k)hy<x)),  whenever  [x- £htfx),x+ £hy<x)]  Cl 
“  hy>k=0 

and  A^f(x)  =  0  otherwise. 

Ditzian  and  Totik  proved  in  Chapter  3  the  equivalence  of  this  modulus  with  the 
modified  K  -functional 

KJ/f,.r)p  =  ini  {||f-g||p+tr||y>2g(r)||p+t2r||g(r)||p ;  g  £  L'(I)> . 


3.  Global  results 


Theorem  3  Let  y<x)  =  >/x(l-x),  n  >  s.  If  f  6  L*(I),  1  <  p  <  »,  then 

IKMnf-0(,)llp  <  C{«2(f<  V1/2)p4^  ¥%>  (14) 

with  a  constant  C  independent  of  n. 


I 


Remarks  (14)  implies  that  1  im  ||(Mnf-f)^||  =  Ofor  every  f  €  L®(I),  1  <  p  <  », 

n-"»  P  P 

since  lim  w“({^s\t)p  =  0  holds  true  for  every  function  of  this  class. 
s+2 

For  f  €  Lp  (I),  1  <  p  <  oo,  (14)  implies  that  the  rate  of  simultaneous 
approximation  is  ||(Mnf-f)^||p  =  0(n’^)  (n-»  oo)  because  in  this  case 

For  the  proof  of  Theorem  3  we  still  need  the  following  lemma  which  can  be  proved 
in  a  similar  way  as  Lemma  3.2  in  Heilmann  and  Mfiller  ( 1989). 


Lemma  4  Let  s  €  iQ,  n  €  M.  n  >  s,  u  €  I  and 

Hn(«)  =  (n+l ){//-//}(»  -t  .Sik(x)pB+a+s(t  )dtdx. 

-1  0 

Then  we  have  with  y)(u)  =  ^u(l-u)  the  estimate  Hn(u)  <Cn*Mu)“, 
where  C  denotes  a  constant  independent  of  n  and  u. 

Proof  of  Theorem  3  In  order  to  prove  (14)  we  make  use  of  the  equivalence 

2 

Using  (3)  and  (8)  we  have  for  every  g  €  Lp(I) 

IKMnl-f)(,)ll  <  C1||(W -«llp+l|M„  sg-g||p.  ( 15) 

We  expand  g  by  the  Taylor  formula  with  integral  remainder 

t 

g(0  =  g(x)+(t-*)g,(*)+Ro(g.t-x)i  R0(g,t,x)=  /(t-u)g"(u)du. 

x 

(13)  and  (12)  imply 

llMniS8-Sllp  s  c2"  ’'lllellp+lle'llpl+IIMp  sR^g.  MXxlllp 

<  C3n'1I||g||p+||A"llpl+l|M„  sR2(g,  ',xXx)||p,  (16) 

where  the  last  inequality  results  from  the  estimate  (a)  in  the  proof  of  Theorem  9.5.3 
by  Ditzian  and  Totik. 

Next  it  will  be  shown  that 

HMn,.R2<8> '  *  C4”  '1II(V’2+"  'VlL  1 1?) 

In  fact  it  is  enough  to  prove  (17)  for  p  =  1  anap  =  «.  Then  the  cases  1  <  p  <  » 
follow  by  the  Riesz-Thorin  theorem.  Utilizing  (12)  with  m  =  1  the  proof  of  (17)  for 


I 


p  =  oo  is  the  same  as  for  (5.10)  in  the  paper  of  Ditzian  and  Ivanov.  For  p  =  1  f  1 71  is 
derived  by  applying  Fubini's  theorem  twice,  then  the  definition  of  Hn(u)  and 
Lemma  4.  This  gives 

]|Mn8(R2(g,-,xXx)|dx 

<(n+IMn,»)^SoPn.sk(x^pn+Sil+s(t)|/(t-u)g"(»)du|dt<li 
=  (n+l)a(nlS)^|g"(u)|{//./^(u-t)ki()PI1.sk(x)pn+5ik+s(l)dtdid» 

<  a(n,a)/|g"(u)|HI1(u)du  <  C5n 'I||st>2g,,)|1  <  C5n'1||(p2+D‘1)g"||1, 

where  is  independent  of  n.  Thus  ( 17)  is  proved.  Together  with  ( 15)  and  (16)  this 
gives 

ll(Mnf-f)<s)llp 

<C1||f(s)-g||p+C3n-1[||(ts>-g||p+||f(s)||p+||#,2g''||p+||(»,2+„-1)g.'||pJ 

-<C{||i(s)-g||p+»'1||p2g',IH-.-2||g"||p+n-1|H<s)||p}. 

Taking  the  infimum  over  all  g  on  the  right  hand  side  leads  to 

IKMnf-0(,)IL  <  C{E2(l<s>,n-1)p+n-1||l<I)||p} , 

which  concludes  the  proof  of  Theorem  3.  □ 

We  did  not  succeed  to  invert  Theorem  3.  Therefore  we  studied  weighted 
approximation.  Our  next  theorem  is  again  a  global  direct  theorem,  but  now  for  the 

rate  of  weighted  simultaneous  approximation  ||p2s(Mnf-f/28)||  .  This  rate  will  be 

estimated  using  the  second  order  weighted  modulus  of  smoothness  w  (f,t)  2s  on 
I,  whose  definition  is  given  by  Ditzian  and  Totik,  Appendix  (B.l).  In  the  proof  we 
use  the  equivalence  of  this  modulus  with  the  following  modified  weighted 
K -functional  on  I 


K2(f,t2)  2s 
9  9  .P 

_  •  t  fii  2s// 


.2h  ,2s  2  " 


=  inf  {||^(f-g)||p+ti^/g  Hp+tWlIp  I  g'€ACloc(I),  /V  €  Lp(I)}  • 

ItomS  ,  n  >  2s.  If  € Lp(I),  1  <  p  <  ®,  then 

|^28(Mnf-f/28)||p  <  Cf*2^28  V1'2)^.  p+n-1||^28>||p}  . 


i 


(0_\ 

For  the  proof  the  representation  ( 1)  for  (Mnf)v“  ’  has  to  be  rewritten  into 

AxfyujP'Xx)  =  (n+l)||s^".aCn,2.,k)plltk+B(x)^pn>k+B(t)»<t)aBrf2B>(t)dt 

provided  ^  l  (I),  1  <  p  <  oo,  n  >  2s,  where 

^n,2s,k)  =  [(( Y^j jl  <  L 

Again  the  most  difficult  step  in  the  proof  is  to  show  that 

||yj2aMn  2sFt2(«>  •-x)llp  *  Cn  '1!k2s(  ^2+n  "1)g"llp-  For  details  of  the  proof  of  this 
and  the  following  two  theorems  see  our  forthcoming  paper ,  where  we  derive  similar 
results  for  the  simultaneous  approximation  by  the  general  method  of  Baskakov- 
Durrmeyer  operators. 

If  ^2S|(2s)  g  jm  ^  =  0(t^a'9)),  0  <  a-s  <  I  (i.e. 

P  ^  ^'p. 

s  <  a  <  s+l),  which  is  equivalent  to  (f»t)p  =  0(t^a)  by  Ditzian  and  Totik, 

Corollary  6.3.2,  then  Theorem  5  implies 

||p2s(Mnf-f/2s)||  =  0(ns_a).  For  s  <  a  <  s+l  this  result  can  be  inverted  by 
using  the  Berens  -Lorentz  lemma. 


Theorem  6  Let  p25^2^  6  Lp(I),  1  <  p  <  a,  i»(i)  =  V*(  1  -x),  n  >  2s,  s  <  a  <  s+l. 
Thee  II  Am„I-1)(2s)|Ip  =  0(as‘“)  implies  «2(s+1)(f,t)p  =  0(t2“) . 

Mainly  as  a  corollary  of  Theorems  5  and  6  the  following  equivalence  result  is 
obtained. 

Theorem  7  Let  f  €  Lp(I),  1  <  p  <  oo,  ^(x)  =  Vx(l-*)>  n  >  2s,  s  <  o  <  s+l.  Then  the 
following  statements  are  equivalent: 


(i)  w2<*+1>((,t)p  -  0(l2a) 

(ii) w^*i<2*)  E  L_(I)and  lit*2” 


(■■) (  Lp(I)  end  ||pJ’(Mnf-f/2*>||p  =  Ofn’'®) 

Especially  for  s  s  0  we  obtain  from  Theorem  7  the  equivalence 

||Mnf-f||  a*  0(n  ~a)  «  u£(f,t)  *  0(t^°),  which  has  been  proved  by  Ditzian  and 


Ivanov,  Theorem  7.4. 


4.  Local  results 


Derriennic  (1979)  proved  the  local  convergence  (cf.  Theorerae  II. 6) 

lim(Mnf)(s)(x)  =  f(s)(x), 
n-*® 

provided  f  is  integrable  and  bounded  on  I  and  s -times  differentiable  at  the  point 
x  €  I.  Our  final  result  is  a  generalization  of  the  Voronovskaja  theorem  proved  by 
Derriennic  (1986). 


Theorem  8  Let  f  €  L*(I),  s  €  iQ,  f^  twice  differentiable  at  a  fixed  point  x  €  (0,1)- 

Then  lim  n(M  f-f)^(x)  =  rT  L<x)2f'(*)l,  where  p(x)  =  Vx(l-x). 
n-*«  n  dxs+1  1  J 


Proof  Define  F(u)  :=  /  f(t)dt .  Then  F'(u)  =  f^8\u)  a.e.  in  I  and 

0 

F'(x)  =  tsXx),  F"(x)  =  f(s+1)(x),  F'"(x)  =  f*S+2)(x), 

as  is  assumed  to  be  twice  differentiable  in  x.  We  consider  the  Taylor  formula 

F(t)  sb  F(x)+(t-x)F'(x)+2(t-x)2F"(x)+j!(t-x)3F,"(x)+(t-x)3R(t-x)) 

where  |R(t-x)|  <Cfort  €  I  and  limR(t-x)  =  0. 

t-*x 

Differentiating  this  formula  with  respect  to  t  leads  to 

fW(t)  =  f(s)(x)+(t-x^s+1\x)+^(t-x)¥s+2)(x)+^{(t-x)3R(t-x)]  a.e.  in  I. 
Together  with  Lemma  1  this  gives 

n(Mnf-f)(s)(x)  (18) 

•  [v<i)212(o+1)-4(.+1)(»+2)]+(>+1Xi+2)]  +  * [^{<t -x)3!^* -x)]](x)}  . 

a»  U  {  ^(n-s+j)- n^n+i+ijj-.n^n+i+j)-1 

=  -sfs+ljn8”1  •  n  (n+l+j)'1+0(n*2) 

j=l 

and  1  im  «  1  we  get  from  (18) 


limn(Mnf-f)(s)(x)  =  -s(s+l)f(s)(x)+(s+l)^s+1\x)+¥3(x)2{(s','2)(x),  (19) 

n-*«  '' 

provided  that 

1  imn-M  [^{(t-x^t -x)]l(x)  =  0  .  (20) 

It  is  obvious  that  ( 19)  can  be  rewritten  in  the  form  of  the  proposition  of  the 
theorem. 

Now  we  look  at  the  remainder  and  prove  (20)  where  we  consider  the  case  s  >  0.  For 
s  =  0  we  refer  to  the  paper  by  Derriennic  (1985).  Integration  by  parts  leads  to 

R*(l):*  n  Mn  s  (j[l(t -i)3R<t -i)]](i) 

-n(»+i)a(i.,s)iropn.5k(i)^(-i)p;+!hl+s(t).(t-I)3R(t.i)dt. 

Using  (6)  and  changing  the  index  of  summation  gives 

R,(l)=n(n+l)(n+s)o(n,s)(kS^pn_3k(i)/pn+j_lk+s(t).(t-x)3R(l-x)dt 

+  k=  .1Pn-».k+l(l)^n+»-l,k+»(,)'('  -i)3r<1  -x)dt} 
n  -s 

=  n(n+l)(n+s)or(n,s)^  (Pn_8|k00-Pn.S|k+1(x)) 

,|pn+s-l1kWtHt'x)3R(l'x)dt’ 

as  we  defined  pn^  =  0ifk<0ork>n.  Using  again  (6)  and  the  relation  (7)  one 
obtains 


rfx)V(x)  =  «(".»)  k^_1Pn -s+U+lW  -(k+! -(" -s+!)*) 


0 

Applying  the  Cauchy -Schwarz  inequality  gives 


K+.-1,  l+.«)(.-x)3R(«-x)d. 


Mi)V(i)|!<C-»4jYpMrtW(i).(i+H»W} 

■kf.1pn-5+1,k+l(l)(]pn+.-I,k+.(,X‘-I)3R<‘-l)<i'}i 

The  term  in  the  first  curly  bracket  equals  (n-s+l)p(x)  by  Lorentz's  formulas 
given  in  his  book,  p.14.  Choosing  e  >  0  there  exists  a  6>  Osuch  that  |R(t-x)|  <  f 
whenever  |t-x|  <  6  and  we  obtain 


i 


W*)Vu)]2<c.nV*r  J*  pn.s+i-k(i)^I)<-/pn+s.1,k+a.1(‘X<-i)6<i* 

+Jll  pn+a-l,k+s-l(1)<‘'x)8dt} 
<C-n3f<1)2{.2-Tlls.li6(x)+|-.TIis.18(x)} 

<C#<x,2jf^), 


where  we  used  Lemma  1  and  Corollary  2.  Thus  for  n  big  enough  we  get 
\if(x)^R  (x)|  <  Ce  which  gives  (20)  as  x€  (0,1). 
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Abstract  This  paper  explores  polynomials  orthogonal  with  respect  to  the  Sobolev  inner 
product 

(/,</) a  :=  J  t  f(x)g(x)dx  +  A^  ^  f'(x)g'(x)dx,  A  >  0. 

We  investigate  expansions  in  Legendre  polynomials — it  transpires  that  their  coefficients 
satisfy  an  explicitly  known  recurrence  relation.  Moreover,  we  re-interpret  a  result  of 
Althammer  [1962]  and  Grobner  [1967]  on  a  differential  relation  which  is  obeyed  by  the 
Sobolev-orthogonal  polynomials  and  exploit  it  to  derive  a  useful  expression  for  the  cor¬ 
responding  Fourier  coefficients.  These  results  lead  to  an  efficient  algorithm  for  approx¬ 
imation  by  polynomials  in  the  underlying  Sobolev  space.  This  algorithm  is  introduced 
and  described  in  detail,  accompanied  by  numerical  examples. 

Key  words:  Fourier  coefficients,  Legendre  polynomials,  Polynomial  approximation.  Re¬ 
currence  relations,  Sobolev  norm,  Ultraspherical  polynomials. 


1.  Orthogonality  in  a  Sobolev  space 

The  theme  of  the  present  paper  is  orthogonality  with  respect  to  the  Sobolev  inner  product 

(f,9) A  :=  J  t  f{x)g{x)dx  +  A  ^  f'{x)g\x)dx,  (1) 

where  A  is  a  non-negative  real  parameter,  while  /  and  g  range  across  the  Sobolev  space 
Wj  (the  set  of  real  functions  with  Lj-integrable  derivatives). 

Each  inner  product  that  acts  on  polynomials  generates  (e.g.  by  the  Gram-Schmidt 
process)  a  set  of  orthogonal  polynomials.  Disregarding  normalization  for  the  time  being, 
we  denote  them  by  PqA\  p[A\ p^,  ■ 


j  1  P(m]{x)P(n){x)dx  +  A  J  ^  P{m)\x)piX)'(x)dx  j  >  J 


:  m  ^  n 


Sobolev  orthogonality  has  been  already  introduced  and  debated  by  several  authors,  in 
particular  Lewis  [1947],  Althammer  [1962]  and  Grobner  [1967], 1  in  a  more  general  setting: 
Let  y5o,^ii  •  •  ■  i^PL  be  L  +  1  given  distributions  (i.e.  real,  right-continuous,  monotonically 
non-decreasing  functions  with  an  infinite  number  of  points  of  increase  and  with  all  mo¬ 
ments  bounded)  on  the  real  interval  H.  One  may  consider  polynomials  orthogonal  with 
respect  to  the  inner  product 

/  f(x)g(x)d<fio(x)  +  f  f'(x)g'(x)difi1(x)  +  ■■■  +  f  f{L)(x)g{L)(x)d(pL(x).  (3) 

Jn  Jn  Jn 

It  is  possible  to  prove  easily  that  these  monic  polynomials  solve  the  variational  isoperi- 
metric  problem 

min  |  y2(x)dip0(x)  +  J ^  y'2{x)d<p i(x)  + - \-  y{L)2(x)d<pL(x)^  , 

where  y  ranges  across  all  monic  polynomials  of  given  degree  [Althammer,  1962],  Some 
of  our  present  results  extend  to  the  inner  product  (3),  and  they  will  be  subject  of  a 
forthcoming  paper.  In  the  present  work  we  focus  on  the  simpler  (and  the  most  useful!) 
form  (1). 

It  follows  readily  from  (2)  that  the  underlying  Fourier  coefficients  of  a  function  /  are 


M  A)  = 


otfUV 


0,1,. 


(4) 


Thus,  if  /  is  an  m-th  degree  polynomial  then  /  =  En=o  /n(A)pnA\  whereas  for  any 
/  €  Wj  it  is  true  that  lim*,^*  ||/  -  En=o/r»(*)P»A>IU  =  0.  Here  ||  -  ||A  is  the  norm 
induced  by  ( • ,  •  )a  [Lewis,  1947], 

A  mechanism  to  approximate  functions  by  polynomials  in  Sobolev  norm  is  useful  in 
numerous  applications,  when  derivatives,  and  not  just  function  values,  are  important, 
e.g.  in  spectral  methods  for  differential  equations.  The  standard  Legendre  projection  (i.e. 
A  =  0)  produces  poor  approximation  to  the  derivative,  which  might  be  pointwise  worse 
by  orders  of  magnitude  than  the  underlying  approximation  to  the  function:  an  example 
to  this  effect  features  in  §4. 

In  §2  we  introduce  a  representation  of  pi**  as  a  linear  combination  of  Legendre  polyno¬ 
mials.  The  coefficients,  which  depend  on  A,  obey  a  known  three-term  recurrence  relation, 
hence  can  be  obtained  easily. 

§3  is  devoted  to  a  differential  relation  that  is  satisfied  by  ph*\  Although  it  has  been 
already  debated,  in  a  different  form,  by  both  Althammer  [1962]  and  Grobner  [1967],  we 
provide  both  easier  derivation  and  a  different  interpretation  of  the  result.  Our  approach 
leads  to  a  relationship  between  the  Fourier  coefficients  (4)  and  the  quantities 

/„*:=  fif(x)Pi1'1\x)dx,n  =  0,1,... 


where  Pn  are  the  (1, 1)  ultraspherical  polynomials  [Chihara,  1978].  We  produce  recur¬ 
sively  the  coefficients  that  feature  in  that  relationship.  Results  of  §2  and  §3  are  assembled 
into  a  numerical  algorithm  to  project  functions  into  polynomials  in  W 2*. 


'We  are  grateful  to  Prof.  W.  Gautschi  for  drawing  our  attention  to  the  above  references. 
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Finally,  in  §4  we  present  a  computational  example  demonstrating  that,  given  a  func¬ 
tion  with  “awkward”  derivative,  the  expansion  in  Legendre  polynomials  produces  poor 
approximation  to  the  derivative,  while  expansion  in  Sobolev-orthogonal  polynomials 
brings  about  good  approximation  to  both  the  function  and  its  derivative. 

Future  papers  will  address  themselves  to  the  theory  of  Sobolev-orthogonal  polynomi¬ 
als  in  a  wider  context. 


2.  Explicit  representation  of  p„ 


,(A) 


As  it  calls  for  no  extra  effort,  we  consider  polynomials  orthogonal  with  respect  to  the 
inner  product  (3)  with  L  —  1.  We  denote  the  polynomials  orthogonal  (in  the  conventional 
sense)  with  respect  to  y?o  by  {gn}  and  say  that  the  distributions  {<po><Pi}  have  property 
a  if  for  every  m,n  =  0,1,...  it  is  true  that  fKq'n(x)q,m(x)d<pi(x)  =  d^^^y  It  will  be 
assumed  henceforth  that  property  a  holds. 

We  seek  7„,o, . . .  ,7n.o  such  that 


P{nHx)  =  ^2ln,k(X)^k(x).  (5) 

k=0 

Denoting  cn  :=  fKq*(x)dip 0(x),  dm  :=  fK  q'n(x)q,m(x)d<p1(x)  (for  m  <  n)  we  have 

n 

0  —  ^ ,  qt^  7n,ncn  4"  A  )  '  T'n,i*4nin{fc,/}*  f  =  0,  l,...,7l  —  1. 

fc=0 

Thus,  given  that  C  :=  diag{c0,Ci, . . .  ,cn},  D  -  (cL.^y, /})  u  and  en  is  the  n-th  unit 

vector,  we  have 

(C  +  XDYt  =  «en,  (6) 

where  7n  =  (7n,o, 7n,i>  •  •  •  i7n,n)T  and  w  ^  0  is  a  normalization  constant,  to  be  chosen 
later  at  our  convenience.  Solution  of  (6)  follows  easily  by  Cramer’s  rule: 


Pn^(z)  =  wdet 


Co  +  Ado 

Ado 

Ado 

Ado 

qo(x) 

Add 

ci  +  Adi 

Adi 

Adi 

qi(x) 

Ado 

Adi 

ci  +  Ad2 

Ad2 

92(x) 

Ado 

Adi 

A  d2 

• •  •  c„_i  +  Ad„_i 

?n-l(x) 

Ado 

Adi 

Ad, 

A  dn_, 

9»(x)  . 

(7) 


where  u>  is,  again,  a  non-zero  constant. 

We  now  subtract  the  bottom  row  of  (7)  from  the  remaining  rows.  This,  in  tandem 
with  7„,o  =  do  =  0  yields 


p\^\x)  =  wdet 


Cl 

A(d,  -  d2)  A(d,  -  d3) 

••  •  A(di  —  dn_i) 

9i(x)-?„(x) 

0 

c2  A(d2  -  ds) 

•••  A(d2-dn_i) 

qs(x)  -  qn(x) 

0 

0  c3 

•••  A(d3  -  dn_,) 

93(x)  -  qn(x) 

0 

0  0 

Cfl-1 

qn-i(x)  -  qn(x) 

.  Adi 

A  d2  A  d3 

Adn_, 

9n(x) 

■  • 
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Observe  that,  when  performing  Gaussian  elimination  to  bring  the  matrix  into  upper 
triangular  form,  the  coefficient  r/  used  to  eliminate  the  £-th  element,  1=1,. ...n— 1,  is 
independent  of  n.  These  coefficients  satisfy 

m  —  1 

^dm  ^  (dm  djc)vje  4*  cmrm  —  0,  T7i  =  l,2,...,7i— 1.  (8) 

k=l 

Our  contention  is  that  the  r*.’s  obey  the  three- term  recurrence  relation 

Ttrt  =  (</  +  u/A)r/_!  -  e<r,_2,  (  =  3,4,...  (9) 


where 


t(  =  ce(d(_i  -  d*_2); 

-  d*_2); 

vt  =  ( dt  -  dt^i)(dt.i  -  dt_2); 

£(  =  ~  dt_i); 

with  the  initial  conditions 

r0  =  0;  rj  =  ——A;  r2  =  -  —  A  -  ^A2. 

Cl  C2  CiC2 

Indeed,  substituting  (8)  into  (9)  yields 


Ttrt  -  («/  +  u/A)i7_!  -f  etrt-2 


=  -A  |(dr-i  -  d<_2)  ^d/  -  ^(d*  -  dfc)r*j  -  (dt  -  dt-2)  ^d/_1  -  Y(dt-i  -  dk)rkj 
+  (di  -  dt- 1)  |d*_2  -  Y^l~2  -  dk)rkj  -f  (dt  -  dt-i)(dt-i  -  d*_2)r/_ij  =  0 


and,  since  both  (8)  and  (9)  possess  unique  solutions,  our  assertion  is  true. 

A  point  of  interest  is  that,  subject  to  consecutive  df  s  being  distinct  and  >  0,  (9) 
implies,  via  the  Favard  theorem  [Chihara,  1978],  that  the  shifted  polynomials  iy(A)  := 
-^r/+1(A),  l  =  0, 1, . . .  are  orthogonal  with  respect  to  some  real  distribution. 

Next,  we  evaluate  for  future  use  X)i_1  Tk-  The  identity  (8)  implies  that 


t-i  t-  i 

A  Y  dkrk  —  ~ctrt  -  deX  +  d( A  Y  rk ■ 

k=l  k=  1 

Shifting  the  index  by  one  and  adding  Ad/_ir/_$  to  both  sides  produces 
t-i  t- 1 

A  Y  =  ~cr-ir*-i  -  d*_iA  +  d(_ jA  Y  rk- 

k=  1  *=1 


(10) 


(11) 


We  now  solve  (10)  and  (11)  for  the  unknowns  rk  and  £i_1  dkrk.  Subject  to  d/  ^ 
dt- 1  this  yields 


We  now  proceed  to  evaluate  the  expansion  of  p^.  To  this  end,  we  fix  the  normal¬ 
ization  constant  ui  so  that  the  coefficients  are  polynomials  in  A  and  pj,0^  =  qn.  Having 
eliminated  the  bottom  row  from  the  determinant,  it  follows  at  once  that 

n— 1 

p!A)(*)  =  9n(x)  +  Y  -  Qn(x)), 

1=  1 

and  (12)  gives  an  explicit  expansion  of  p ^  in  qt' s, 

PnA,(*)  =  -*n(A )qn(x)  +  Y  rt(\ )qt(x).  (13) 

t=  l 

An  interesting  identity  readily  follows  from  the  expansion:  Subtract  (13)  from  the 
corresponding  expression  for  p^x.  This  yields 

Pnh(x)  ~  PnHx)  =  ~crn+i(X)qn+l(x)  +  (rn(A)  +  crn(A))9n(i). 

However,  since  (12)  implies  that  <7n+x(A)  =  rn(A)  -(-  <rn(A),  we  obtain 

P{n+i(x)  -  P{n\x)  =  -<Tn+i(A)(gn+i(i)  -  q„(x)).  (14) 

Identity  (14)  is  useful  in  the  explicit  evaluation  of  the  p^’s. 

An  example  of  distributions  that  obey  property  a  is  the  Laguerre  pair  dip0(x)  = 
dipi(x)  =  T^\o^xae-zdx,  where  0  <  x  <  oo  and  a  >  -1.  It  can  be  proved  that 

d(  =  ^^7*  and  that,  for  a  =  0,  the  ry’s  can  be  identified  with  Chebyshev  polynomials 
of  the  second  kind. 

Legendre  weights  do  not  satisfy  property  a  unamended.  Fortunately,  since,  in  that 
case,  both  pi-^  and  qn  =  P„  maintain  the  same  parity  as  n,  property  a  is  “recovered”  as 
long  as  attention  is  restricted  to  indices  of  the  correct  parity — the  remaining  coefficients 
vanish  and  are  of  no  interest!  Since,  integrating  by  parts, 

£  Pm(x)K(x)dx  =  P;(l)Pn(l)  -  P;(-1)P„(-1)  -  £  K(x)Pn(x)dl, 

it  follows  readily  that  (subject  to  the  aforementioned  restriction  on  parity)  d(  =  £(  1  +£). 
This,  together  with  ct  =  3^,  (9)  and  (13)  implies  that 

[a=l] 

Pri^  =  —(7nPn  +  rn-2iPn-2ti  (15) 

*=1 

where 

=  (2  +  (21  -  1)(2 1  -  5)\)rt_2  - 

_  rn-2 
a  =  2n+1  2n~3 

n  2(2n  -  1)A  ’ 

with  the  initial  conditions 


■  • 


i 


ro(A)  =  0;  rx(A)  =  -3A;  r2(A)  =  -15A;  r3(A)  =  -42A  -  105A2. 
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3.  A  differential  equation 


We  restrict  our  attention  in  the  present  section  to  the  Legendre  distribution.  Let  t  be  an 
arbitrary  polynomial  of  degree  <  n  -  3.  Obviously,  ,  ( 1  -  x2)tj  ^  =  0.  Integration 

by  parts  produces 

J  i 1  ~  x2)  ( Pn](x )  -  Xp(nx)"(xj)  t(x)dx  =  0. 

Hence,  p^  -  A ,p£^  is  orthogonal  (with  respect  to  the  distribution  (1  -  x2)dx)  to  all 
polynomials  of  degree  <  n  -  3  and,  taking  parity  into  account,  there  exist  q„(A)  and 
/3„( A)  so  that  the  Sobolev-orthogonal  polynomial  obeys  the  oiuinary  differential  equation 

P{nX\x)  -  A— ptA>(«)  =  a„(A )PW(x)  +  /?„(A)/><l_f  (x),  (16) 

where  the  pj^'^'s  are  ultraspherical  polynomials  [Chihara,  1978]. 

The  importance  of  (16)  becomes  apparent  upon  the  consideration  of  the  Fourier 
coefficients.  Integrating  by  parts  we  have 

/nlb!A)||2  =  /‘  fix)  {p(X\x)  -  A p<A>"(x))  dx  +  A  {p<A>'(l)/(l)  -  piA),(-l)/(-l)} 


=  «»/„*  +  /W„*-2  +  A  {plA)'(l)/(l)  -  plA)'(-l)/(-l)}  ,  (17) 

where  fa  :=  / -l1  f(x)Pn’1\x)dx,  m  —  0, 1, ...  Since  the  values  of  plA^  and  its  derivative 
at  the  end-points  can  be  easily  derived  from  the  results  of  §3,  it  remains  to  provide  an 
explicit  form  of  an  and  0n. 

Comparing  coefficients  of  xn  in  (15)  and  (16)  yields  at  once 

_  n  +  2 
a"  ~  ~2(2n+l)<Tn’ 

It  is  an  easy  exercise  to  demonstrate  that 

/>■>(**.- 2(1 +  (-in;  (IS) 

J-i  m  +  2 


2(i  +  (~ir+i) 

m  +  2 


(19) 


Let  n  be  even.  Then  integration  of  (16)  from  —1  to  1,  in  tandem  with  (18),  yields 


n  +  2  n 


Likewise,  for  odd  n,  we  multiply  (16)  by  x,  integrate  and  employ  (19):  Since,  by  (17) 
and  integration  by  parts  respectively, 


jxxiX\x)dx  =  |  ri; 
j  x  xpW"(x)dx  =  2(pW'(l)-p|lA)(l)), 


we  obtain 


Legendre  projection  and  its  derivative  Sobolev  projection  and  its  derivative 

Figure  1:  Legendre  and  Sobolev  projections. 

The  thicker  line  denotes  the  function  /, 
whereas  the  thin  lines  stand  for  polyno¬ 
mials  of  degrees  3, 6, ,  18. 


All  the  ingredients  for  an  algorithm  for  the  evaluation  of  a  projection  of  /  into  n-th 
degree  polynomials  in  are  now  in  place:  either  (14)  or  (15)  are  used  to  evaluate 
Pm*’ s,  the  quantities  /„,  which  are  independent  of  A,  are  evaluated  e.g.  by  numerical 
quadrature  and  substituted  into  (17),  whereas  an  and  0n  are  also  available  from  explicit 
formulae  of  this  section.  The  norms  |jpnA*||J  can  be  evaluated  either  directly  from  the 
explicit  form  or  via  a  recurrence  relation  which  is  omitted  here  due  to  space  constraints. 


The  satisfaction  of  an  “interesting”  ordinary  differential  equation  by  p^  has  been 
already  noticed  by  Althammer  [1962],  who  obtained,  in  a  considerably  longer  derivation, 
the  right-hand  side  in  terms  od  derivatives  of  Legendre  polynomials,  and  by  Grobner 
[1967],  who  identified  it  with  the  n-th  derivative  of  a  Lagrange  multiplier  function  of  the 
isoperimetric  variational  problem  from  §1.  However,  none  has  used  it  to  facilitate  the 
evaluation  of  Fourier  coefficients.  Moreover,  our  method  yields  itself  to  other  distributions 
as  well.  The  characterization  of  all  such  distributions  wiil  be  described  in  a  forthcoming 
paper. 


4.  A  numerical  example 

The  approach  of  this  paper  becomes  valuable  when  we  wish  to  approximate  a  function  / 
by  its  projection  into  polynomials  and,  simultaneously,  to  approximate  its  derivative  by 
the  derivative  of  the  polynomial  approximant.  Given  that  the  derivative  of  /  is  steep,  it 
is  only  to  be  expected  that  the  quality  of  the  projection  in  the  conventional  Li  norm  (the 
Legendre  projection)  deteriorates.  Several  computational  examples  show  that  in  this  case 
the  Sobolev  projection  is  superior.  Figure  1  displays  approximants  of  both  types  (with 
A  =  0  and  A  =  ^  respectively)  to  the  function 

f(x)  =  e-100^)2,  -1  <  ar  <  1, 

by  polynomials  of  degrees  n  =  3,6, 9, . . . ,  18.  It  is  evident  that  the  Legendre  projection 
is  poor  near  the  end-points,  whereas  the  Sobolev  projection  displays  reasonably  good 
behaviour  throughout  the  interval. 
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PIECEWISE  POLYNOMIAL  APPROXIMATION  OF  POLYNOMIAL  CURVES 


M. A.  LACHANCE 

Department  of  Mathematics 

University  of  Michigan  -  Dearborn 


Abstract  An  algorithm  based  upon  Chebyshev  economization 
and  constrained  Chebyshev  polynomials  is  discussed  for 
producing  low  degree  piecewise  polynomial  approximations 
for  high  degree  polynomials.  The  method  is  shown  to 
compare  favorably  to  the  Remez  algorithm. 

Key  words  Constrained  Chebyshev  polynomials,  Chebyshev 
economization,  Remez  algorithm 


1 .  Introduction 

The  majority  of  European  and  Asian  computer-aided  design 
(CAD)  systems  use  parametric  polynomials  to  represent  3D 
curves  and  surfaces.  These  systems  differ  in  their 
internal  representation  of  polynomials  (power,  Bernstein, 
and  Hermite  bases)  and  in  their  maximum  degree  (three, 
seven,  nine,  fifteen,  and  twenty-one).  In  order  for  these 
systems  to  exchange  geometric  data,  representations  must 
occasionally  be  altered,  and/or  degrees  must  be  reduced. 

The  numerical  stability  of  altering  representations  has 
been  recently  investigated  by  Heybrock  (1987)  and  Shearer  J 


C'  CL- 


I 


LG 

■  i 
•  n 

14 

15 

1 6 

18 

19 


25 

26 

27 

28 

29 

30 

31 
3  2 
3  3 
34 
3  5 

36 

37 
3  8 

39 

40 

41 

42 

43 

44 

45 

46 

47 

48 

49 

50 

51 

52 

53 


(1988) .  In  this  note  we  are  concerned  with  the  problem  of  ! 
degree  reduction. 

In  general,  a  high  degree  polynomial  will  be 
approximated  rather  poorly  by  a  lower  degree  polynomial, 
and  hence  piecewise  or  spline  approximates  are  required. 

We  shall  describe  a  piecewise  polynomial  construction  which 
is  recursive,  is  uniformly  near-best,  controls  geometric 
continuity,  and  is  relatively  inexpensive.  The 
jconstruction  is  compared  with  the  Remez  algorithm. 

2.  Constrained  Chebyshev  polynomials  1 

! 

Lachance,  Saff  and  Varga  (1979)  introduced  the  collection 
of  constrained  Chebyshev  polynomials  as  a  solution  to  a 
minimax  problem  with  Jacobi-type  weights.  These 
polynomials,  denoted  by  T^a'^x) ,  are  the  unique  monic 
polynomials  which  are  extremal  for  i 

Problem  2.1  For  each  triple  of  nonnegative  integers  j 

(m,a,/9),  determine  1 

minimum  maximum  (l-x)a(l+x)0|x®  -  g(x)  |  .  j 

-1<x<1  j 

Here,  wm  denotes  the  collection  of  real  polynomials  of 
degree  at  most  m,  and  *  (0). 

i 

These  constrained  Chebyshev  polynomials  are  a  proper 
generalization  of  the  classical  Chebyshev  polynomials  (of 
j  the  first  kind)  since  2m“1T^0, °(x)  -  Tffl(x) .  Most  CAD 
I  systems  parametrize  curves  on  the  interval  [0,1].  We 
shall  do  the  same,  and  introduce  more  convenient  notation 
for  the  constrained  Chebyshev  polynomials  on  this  interval. 

I 

Definition  2.1  For  each  pair  of  nonnegative  integers 
(m,a),  with  m  >  2a,  the  constrained  Chebyshev  polynomials 
|  are  defined  by 
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Thus  m  denotes  the  degree  of  the  polynomial  and  a 
denotes  the  order  zero  at  the  endpoints  0  and  l. 


3 .  The  construction 


Let  P(s)  e  rj, 

of  degree  n  in  R  and  let  E  denote  a  prescribed 
tolerance.  We  wish  to  construct,  within  this  tolerance,  « 
visually  smooth  piecewise  polynomial  approximate  to  P(s) 
of  degree  m.  We  do  so  applying  Chebyshev  economization 
(CE)  in  a  component -wise  manner. 

Let  k  *  n  -  m,  and  define  the  sequence  of  polynomials 
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*i+1(s)  pi(s>  “  cS-i(s> 

1  A  (n-i)l 


where  P0(e) 

polynomials  was  suggested  by  Lachance  (1988)  in  order  to 
control  geometric  continuity.  A  pointwise  bound  on  the 
error  between  the  original  polynomial  and  the  final 
approximate  Pk(s)  is  given  by 


Jg  En- 

1-0  tn-i) I 


i* 


where  |*|E  denotes  the  Euclidean  norm  in  R3.  Clearly, 
if  the  above  error  exceeds  the  prescribed  tolerance  E, 
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then  the  approximation  is  unacceptable.  In  this  event  the 
original  curve  may  be  partitioned  into  two  or  more  pieces ;  :: 
the  pieces  reparametrized  to  the  interval  [0,1];  and  then  .  ' 
the  construction  may  be  applied  to  each  piece  (see  Shearer  4  8 
(1987)). 

Since  there  is  no  a  priori  indication  that  the  final  45 
error  will  be  satisfactory,  the  difficulty  that  arises  is i  ^ 

-t  Z 

how  one  should  break  up  the  original  curve.  Simple  42 

bisection  leaves  two  polynomial  curves  which  must  be  ^ 

approximated,  again  with  no  prior  knowledge  that  the  39 

reduction  scheme  will  be  successful,  perhaps  leading  to  a> 
further  refinement  of  P(s).  A  more  efficient  approach  is  3  6 
to  force  the  errors  at  each  stage  to  fall  below  some 

threshold,  say  32 

3  2 

Mpi+l(s)  -  *i(s)||E  *  *n-L  -  30 

(n-i) !  29 

28 

for  0<iSk-l.  With  the  stepwise  errors  so  bounded,  the 
accumulated  error  will  always  be  less  than  E.  If  the  25 
stepwise  error  should  exceed  this  bound  at  any  step,  then  ^ 
the  polynomial  Pj^s),  instead  of  P(s) ,  can  be  split  and1  22 
reparametrized  into  Pi(s/2)  and  Pi((l+s)/2),  se[0,l].  j 
This  latter  approach  involves  maintaining  a  stack  of  19 

polynomials  to  be  economized,  and  has  the  advantage  of  ! 
adaptively  refining  the  original  curve.  16 

No  matter  which  partitioning  scheme  is  used,  the  '  ^ 

geometric  continuity  of  the  final  piecewise  approximate  i4  13 

'  1  2 

controlled  by  the  value  of  the  parameter  a.  The  11 

approximate  will  be  continuous  if  a»l,  tangentially  smoothl 0 

Q 

if  a»2,  and  curvature  continuous  if  a-3.  This  is  g 

assured  by  the  fact  that  each  iterate  Pi(s)  matches  7 

tangent  and  acceleration  directions  where  joined,  depending  c 
on  the  value  a.  That  is,  4 


a 


0, . . . ,a-l;  S 


0,1. 


For  a  complete  discussion  of  geometric  versus  parametric 
continuity  we  refer  the  interested  reader  to  Bartels, 
Beatty  and  Barsky  (1987). 


4.  Chebyshev  Economisation  and  the  Remes  Algorithm 

At  the  heart  of  the  preceding  section  is  constrained 
Chebyshev  economization.  It  is  natural  to  ask  how  this 
construction  compares  with  existing  methods,  and  can  it  be 
improved  upon.  In  this  section  we  make  some  qualitative 
observations  about  this  approximation  process,  and  compare 
this  method  to  an  implementation  of  the  Remez  algorithm. 


4.1  Chebyshev  Economisation 

■  0 

The  Chebyshev  expansion  for  a  function  on  an  interval  has  29 

long  been  a  tool  of  numerical  analysts.  The  truncation  of  28 

27 

that  series  approximates  the  function  almost  uniformly,  2e 

with  an  error  on  the  order  of  the  first  neglected  term.  •  25 

4-K  24 

Thus  it  is  not  difficult  to  see  that  if  an  ntn  degree  23 
polynomial  is  approximated  using  CE  by  a  constant  22 

1  21 

polynomial,  the  coefficients  in  the  degree  reduction  20 

process  are,  in  effect,  the  coefficients  of  the  i  19 

i  1  8 

polynomial's  Chebyshev  expansion.  Consequently,  17 

terminating  the  process  at  an  intermediate  stage  produces 
near  uniform  approximation  to  the  original  polynomial.  14 

The  cost  in  arithmetic  operations  to  produce  an  m^1  I  1 9 
degree  polynomial  approximate  to  an  n^1  polynomial  is  n 
(n2-m2+n-m) .  If  a  breakup  of  the  original  curve  is 
required,  then  this  cost  times  the  number  of  pieces  8 

provides  an  upper  bound  on  the  total  number  of  operations.  ‘ 
This  cost  can,  of  course,  be  reduced  if  the  original  curve  5 
is  refined  adaptively,  as  suggested  in  the  previous  9 

section.  2 
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0 


4.2  The  Ran  Algoritha 

The  Remez  algorithm  is  a  standard  method  for  estimating  thjeSO 
best  uniform  polynomial  approximate  of  a  continuous  ^ 

function.  It  exploits  the  fact  that  the  error  between  a  4? 
function  and  its  best  mtn  degree  polynomial  approximate  i  ^ 
attains  its  maximum  (in  absolute  value)  at  least  m+2  j  44 
times,  and  does  so  while  alternating  signs.  We  will  not  i  ^ 
discuss  the  method  completely  here,  but  only  summarize  soiite41 
of  the  computational  steps  involved.  Details  can  be  found  ^ 
in  Davis  (1963).  |  38 

Let  f (s)  denote  a  real  function  to  be  approximated  J  IJ 

111+1  I  ^ 

on  an  interval  [a,b],  and  let  (Si)”^  be  an  initial  |  35 

alternation  set.  A  polynomial  p(s)  c  and  an  error  E  ^ 

are  determined  so  that  f(si)-p(si)  *  (-1)^,  l<i<m+2.  Thie3  2 

location  of  a  maximum  of  |f(s)-p(s)|,  say  s*,  is  ^ 

determined  and  exchanged  with  an  appropriate  element  of  29 

(s^jj^J.  The  entire  process  is  repeated  with  the  new  1  ^ 

alternation  set  until  the  actual  maximum  is  satisfactorily  26 

I  OC 

close  to  the  computed  value  E.  !  24 

Each  iteration  of  the  Remez  algorithm  requires  a  number  23 
of  calculations.  If  f(s)  c  :rn,  then  the  initialization  olf^ 
the  m+2  dimensional  system  of  equations  requires  20 

I  1  Q 

(m+2) (2n+m(m-l)/2)  operations,  the  solution  of  this  system^ 
of  equations  requires  on  the  order  of  2(m+2)3/3  j  17 

!  ^  g 

operations,  and  the  difficulty  of  estimating  the  maxima  ofj 
| f (s) -p(s) |  depends  on  the  relative  precision  required,  j  14 
Some  computational  savings  can  be  derived  from  the  fact  |  , 2 
that  the  coefficient  matrix  need  not  be  entirely  redefined  11 
at  each  stage,  but  solution  of  the  resulting  system  and  ths1^ 
search  for  maxima  can  be  costly.  8 

To  more  directly  compare  CE  with  the  Remez  algorithm,  \ 

consider  an  example  where  n-15  and  m-7 .  a  single  pass  8 

4 

through  the  CE  process  takes  184  arithmetic  operations.  3 

The  initialization  of  the  system  of  equations  for  the  Reme 5  2 


Let  f (s)  denote  a  real  function  to  be  approximated 
on  an  interval  [a,b],  and  let  be  an  initial 

alternation  set.  A  polynomial  p(s)  c  and  an  error 
are  determined  so  that  f(s^)-p(s^)  *  (-lJ^-E,  l<i<m+2. 
location  of  a  maximum  of  |f(s)-p(s)|,  say  s*,  is 
determined  and  exchanged  with  an  appropriate  element  of 
(s^jj^J.  The  entire  process  is  repeated  with  the  new 
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algorithm  requires  459  arithmetic  operations,  to  say  i 
nothing  of  the  solution  of  the  9x9  system,  or  the  search 
for  the  maxima. 


5. 


Zolotareff  Economisation 
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To  investigate  this  possibility  consider  the  real  valued34 

'  ,3  3 


Examination  of  the  CE  algorithm  shows  that  it  is  a  rather 
simple  process  involving  only  the  coefficients  of  a  set  ofj 
fixed  real  polynomials,  and  that  the  degree  of  the 
polynomial  is  reduced  by  one  in  each  iteration.  It  seems 
reasonable  to  seek  a  greater  degree  reduction  with 
comparable  arithmetic  simplicity. 


p2(s)  by 


polynomial  p(s)  =  asn  +  aas1*"1  +  *•*  c  ir  Conventional  qE 

-J  £. 

would  have  us  define  the  first  two  approximates  Pl(s)  and!  31 

30 
29 
i  28 
27 
■  26 
;  25 

24 

23 


Pl(s)  *  p(s)  -  a  cn(s)  € 

P2(s>  *  Pi(s)  -  a  (a  +  n/2)  C^fs)  e  irn_2 


Alternatively ,  we  could  define  an  approximate  q(s)  by 
q2(s)  =  p(s)  -  a  Zn(a;s)  e  *n_2,  where 

Zn(o;s)  *  sn  +  as11"1  +  ***  e  rn; 

max  |Zn(o;s)|  «  min  max  |sn  +  asn_1  -  g(s) | 

j  S€[0,1]  9«»n-2  S€ [°> !] 

|  The  extremal  polynomials  Zn(a;s)  were  studied  by 
j  Chebyshev's  student  Zolotareff  (but  with  the  problem 
phrased  on  the  interval  [-1,1]). 

Zolotareff  showed  that  Zn(o;s)  could  be  explicitly 
determined  for  only  a  small  interval  of  a  values;  and 
i  Kirchberger  developed  a  first  order  approximation  to 
Zn(o;s)  for  values  outside  that  interval  (see  Neinardus 
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(1967)).  In  the  explicit  case  the  extremal  polynomial  can  i  i 
be  determined  by  a  translation  of  the  Chebyshev 
polynomials.  In  effect,  ^ 


Zn(a;s)  =  sn  +  asn_1  +”|*  a^a)  s1, 

i 

where  the  coefficients  a^fa)  are  polynomials  in  the 
variable  a.  Otherwise  Zn(c;s)  can  be  approximated  by  a  i 
linear  combination  of  Tn(s)  and  Un(s),  the  Chebyshev 
polynomials  of  the  first  and  second  kind,  respectively. 

The  arithmetic  complexity  of  using  either  of  these 
representations  in  an  economization  scheme  is  at  best  no 
better  than  two  iterates  of  conventional  economization.  Asi 
a  result  it  would  seem  that  CE  is  in  some  sense  optimal 
relative  to  the  quality  of  the  result  and  the  arithmetic 
computations  required. 
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CALCULATION  OF  THE  ENERGY  OF  A  PIECEWISE 
POLYNOMIAL  SURFACE 


EVALD  QUAK 
LARRY  L.  SCHUVAKER 

Department  of  Mathematics 
Vanderbilt  University 


Abstract:  This  paper  is  related  to  the  problem  of  constructing  a  smooth  piecewise 
polynomial  surface  fit  to  data  given  at  the  vertices  of  a  tri angulation.  We  are  inter¬ 
ested  in  methods  in  which  the  surface  is  represented  in  Bernstein- Bezier  form,  and 
the  coefficients  are  chosen  so  that  the  resulting  surface  minimizes  some  expression 
involving  the  energy  of  the  surface  (for  example,  see  [1,3, 5-9]).  Here  we  do  not  go 
into  the  details  of  such  methods  themselves,  but  instead  concentrate  on  the  prob¬ 
lem  of  representing  the  energy  associated  with  a  particular  choice  of  coefficients. 
The  formulae  developed  here  will  be  applied  elsewhere  to  specific  interpolation  and 
smoothing  methods. 

Keywords:  multivariate  splines,  data  fitting,  interpolation,  energy 


1.  Introduction 

Let  A  be  an  arbitrary  triangulation  of  a  domain  fi  C  1R2.  Given  integers 
0  <  r  <  d,  we  define  the  associated  space  of  polynomial  splines  of  degree  d  and 
smoothness  r  by 


Srd( A)  =  {s  G  Cr(fl)  :  s\Ti  €Vd,  i  =  1, . . . ,  N}  ,  (1) 

where  Vd  is  the  (d^2)  dimensional  linear  space  of  bivariate  polynomials  of  total 
degree  d ,  and  where  T,,  i  =  1, . . . ,  N,  are  the  triangles  of  A.  While  the  constructive 
theory  of  these  spaces  of  splines  is  not  yet  complete,  they  are  clearly  useful  tools 
for  fitting  surfaces  to  data  of  the  form 

=  f(xi,Vi)  +  £,,  *  =  (2) 
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where  y*),  i  =  1, . . . ,  V,  are  given  points  scattered  in  the  plane  (and  forming  the 
vertices  of  the  triangulation),  and  where  e*,  i  =  1, . . . ,  V,  axe  measurement  errors. 

In  this  paper  we  Eire  interested  in  two  classes  of  surface  fitting  methods  using 
the  space  55(A).  The  first  class  of  methods  involves  minimizing  some  measure  of 
smoothness  J(u)  over  the  subset 

U  =  {ueSj(A)  :  u(xi,yi)  =  Zi,  t  =  l,...,V}.  (3) 

The  set  U  is  the  set  of  interpolating  splines.  We  may  call  these  kinds  of  methods 
smoothest  interpolation  methods. 

The  second  class  of  methods  of  interest  involve  minimizing  a  combination  of 
smoothness  and  goodness  of  fit  such  as 

Pa(s)  =  A  J(s)  +  E(s), 

over  55(A),  where  A  >  0  is  a  smoothing  parameter,  and  where  E(s)  measures  the 
goodness  of  fit.  For  example,  we  might  take 

v 

E(s )  =  22 

i=l 

This  is  an  example  of  a  penalized  least  squares  method  (see  [6]  in  this  volume). 

While  other  energy  expressions  can  be  treated  by  similar  methods,  throughout 
the  remainder  of  this  paper  we  restrict  our  attention  to  the  functional 

N 

=  I>T,(«),  (4) 

i=l 

where 

JTt(u)  =  J  J[(uxx)2 +2(ux,)2 +(u99)2]dxdy,  i  =  l,...,N, 

Ti 

where  uxx  represents  the  second  order  derivative  of  u  in  the  x  direction,  etc.  This 
expression  represents  the  energy  of  a  ’’thin  plate”,  and  is  a  natural  way  of  measuring 
the  roughness  of  a  surface  (see  [3,9]).  We  have  defined  J  in  (4)  as  a  sum  of  integrals 
over  the  individual  triangles  of  A  in  order  to  be  able  to  apply  it  to  spline  spaces 
55(A)  even  if  the  global  smoothness  r  is  less  them  2.  (Indeed,  in  applications  it  is 
common  to  use  C1  splines,  where  r  =  1). 

The  main  purpose  of  this  paper  is  to  find  formulae  for  representing  J(u)  in 
terms  of  the  coefficients  of  the  Berastein-Bezier  representation  of  a  spline  u  in 
5j(A).  This  is  an  essential  step  in  deriving  numerical  algorithms  for  calculating 
spline  fits  as  discussed  above.  The  remainder  of  this  paper  is  organized  as  follows. 
In  Section  2  we  introduce  the  necessary  notation  for  representing  splines  in  55(A) 
in  Bernstein- Bezier  form,  and  introduce  certain  energy  expressions.  In  Section  3 
we  derive  convenient  recursion  formulae  for  these  energy  expressions.  In  Section  4 
we  apply  our  results  to  give  explicit  formulae  for  cubic  and  quartic  splines. 
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2.  The  Bernstein-Bezier  representation 

In  working  with  55(A)  numerically,  it  will  be  useful  to  regard  SJ( A)  as  the 
linear  subspace  of  52(A)  defined  by 

55(A)  =  {s  G  5d°(A)  :  5  G  Cr(fi)}.  (5) 

Each  spline  s  G  52(A)  can  be  written  in  the  form 

s(x,y)  =  st(x,y)  for  (x,y)€Tt,  l=l,...,iV, 


where  si  is  a  polynomial  of  degree  d,  l  =  1, . . . ,  N.  Each  of  these  polynomials  can 
be  written  in  Bernstein-Bezier  form  as 


s,(r,s,<)=  ^  cijk 

i+j+k=d 


d\rlsHk 
i\j\kl  ’ 


where  (r,  s,t)  are  the  barycentric  coordinates  of  a  point  (x,y)  in  the  triangle  T/. 
Identifying  the  Bezier-ordinates  cll]k  on  common  edges  of  triangles  forces  the  con¬ 
tinuity  of  the  associated  piecewise  polynomial  function. 

With  each  Bezier-ordinate  c\-k  we  associate  a  domain  point 

P'ik  =  W+ivi  +  kvft  /i, 

where  V/,  K,',  and  Vj  denote  the  vertices  of  the  triangle  7).  We  omit  the  superscript 
l  whenever  this  will  cause  no  confusion.  The  set  of  all  domain  points  is  denoted 
by  Bd( A).  On  each  triangle  T  of  A  there  are  precisely  (d  +  l)(d  +  2)/2  points  of 
Bd(  A)  spaced  uniformly  over  T.  It  is  common  to  associate  the  coefficient  c\jk  with 
the  domain  point  Pf]k-  Clearly,  the  set 

C  =  {cfji},  £  —  1  and  i+j  +  k  =  d 

of  coefficients  uniquely  defines  a  spline  in  52(A).  It  is  easy  to  see  that 


nc=#(C)  =  V'  +  (d-l)£+('i2  ‘V 

where  V  is  the  number  of  vertices  in  A  (which  is  also  the  number  of  data  points), 
and  where  E  is  the  number  of  edges  and  N  the  number  of  triangles  of  A. 

It  is  well-known  (cf.  [2,4])  that  a  spline  s  G  52(A)  lies  in  55(A)  if  and  only 
if  an  appropriate  set  of  continuity  conditions,  each  of  which  can  be  described  by 
a  linear  equation  involving  the  Bernstein-Bezier  coefficients,  is  satisfied.  Thus,  we 
can  write 

55(A)  =  {s  G  52(A)  :  Ac  =  6},  (6) 

where  c  is  the  coefficient  vector  of  length  nc,  A  is  an  appropriate  m  x  nc  matrix 
(where  m  denotes  the  number  of  continuity  conditions  enforced),  and  b  is  an  ap¬ 
propriate  m  vector.  The  exact  nature  of  A  and  b  is  of  no  concern  here,  but  is, 
of  course,  critical  in  implementing  a  specific  surface  fitting  method.  We  remark 
that  in  general,  nc  >  V  +  m,  so  that  even  after  specifying  both  smoothness  and 
interpolation  conditions,  there  remain  a  certain  number  of  free  parameters  to  be 
used  in  the  minimization  process. 
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3.  Formulae  for  the  energy  of  a  single  patch 

In  this  section  we  give  formulae  for  the  energy  of  a  polynomial  of  degree  d 
described  in  Bernstein-Bezier  form  on  a  triangle  T.  Throughout  this  section  we 
suppose  that  the  polynomial  is  written  in  the  form 


d  i 

p(r,s,t) 

i=0  j= o 


(7) 


where,  in  general 


(a  +  f3  +  7  )\ras^ty 

a!  /?!  7! 


(8) 


Now  since  p  €  'Pd,  it  follows  that  pxx,  pxy,  and  pyy  belong  to  Vd- 2  and  that 
(D2tx  +2 D2xy+D2yy)p  belongs  to  V1(d-i)-  Introducing  the  notation  b  :=  (61 , bn)T , 
where  n  =  2) ,  we  find  that  the  energy  of  p  over  the  triangle  T  is  given  by  the 

quadratic  form 

JT{p)  =  bTE(d>b, 


where  E is  an  appropriate  symmetric  n  x  n  matrix. 

The  remainder  of  this  section  will  be  devoted  to  finding  the  entries  of  E^d\  We 
determine  them  recursively,  stinting  with  the  case  d  =  2.  As  we  are  dealing  with 
a  rotation  invariant  energy  expression,  without  loss  of  generality  we  can  assume 
that  the  triangle  T  is  in  the  canonical  position  shown  in  Figure  1  with  vertices 
Vx  =  (6,  c),  Vi  —  (0,0),  and  V3  =  (a,  0).  In  this  case,  the  barycentric  coordinates 
( r,s,t )  of  a  point  ( x,y )  are  given  by  the  formulae  r  =  y/c,  s  =  \  —  x / a  +  (b  —  a)y / ac, 
and  t  =  xja  —  by/ac. 


V 


Figure  1.  The  canonical  triangle 


Now 


prr{r,s,t)  =  d(d- 

i=0  j=0 

with  similar  expressions  for  the  other  second-order  partial  derivatives.  Using 

Pit  —  PrrT z  "I"  ^Pra^x^x  d"  ‘^•Prtl'z^x  d"  Paa&x  d*  2pat$xtx  d"  Ptttz  (9) 

and  the  analogous  expressions  for  pxy  and  pyy ,  we  find  that  E^/2ac  is  a  symmetric 
6x6  matrix  whose  entries  in  the  upper  triangular  part  are  as  follows: 


'(RR)1  2RRRS  2RRRT 

ODD  CC  1  O/  D  O  \2  ODD  C'T’  1  O  D  < 


(KS)J 

2RS  RT 

(RT)1 

2RSSS 

2SSRT  +  2  RSST 

2  RTST 

2RS  ST 

2TT  RS  +  2 RTST 

2RTTT 

(SS)1 

2  ST  SS 

(ST)1 

2SSTT  +  2  {ST)2 

2STTT 

(TT)1 

where  R  =  (rx,ry),  S  =  (sx,sy),  and  T  =  (<x,f„),  each  of  which  can  be  expressed 
in  terms  of  the  quantities  a, 6  and  c  describing  the  locations  of  the  vertices  of  T. 

We  now  present  the  main  result  of  this  paper,  a  recursion  relation  for  computing 
the  entries  in  the  matrix  E for  arbitrary  d  >  2. 

Theorem  1.  Let  T  be  a  canonical  triangle  as  in  Figure  1.  Then  for  i  =  0, 1, . . . ,  d, 
j  =  0, 1  k  =  0, 1, . . . ,  d,  and  l  =  0, 1, . . . ,  k, 

Ei£±ll+Hli^+I+1  =  d  ~1)2  J  J  e\dlkA*,y)dxdy, 


where 


_  „2  Ad) 
ei,j,k,l  ei,j,k,l 

+  r4*\dlk-i,,+e\-ij,kA 

+  rt[e\dJ1J_lkl  +  e\dlk_lil_l] 
d-s2eji\  iiifc_ifJ 

+  +  eS-i  j-i.ft-i.il 


These  recursions  begin  with  the  values 

e(2)  ~—E(2) 

~  2ac  ^+2+ i.^+i+i’ 
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contained  in  the  matrix  E(-2'>  given  above. 

Proof:  We  proceed  by  induction.  Since  the  area  of  T  is  ac/2,  the  assertion  is  clear 
for  the  case  d  =  2.  To  prove  the  result  for  d  >  2,  we  use  the  easily  verified  recursion 
formula 


0  =  r$«_i  5,  t)  +  3*ar,J-lt7(r,  S,  t)  +  7-i  (r,  5,  t).  (10) 

Now  by  (9), 


rf  « 

Pxx(x ,  y)  =  —  1)  53  5Z  y)» 

1=0  j=0 

where 

^1  —  1*  z*^  d — i — 2,x — J >}  "t*  — t — l,i — j —  l,j  “f  — t — l,s — j,j — 1 

2,j  +  2sxtz  *&d—i,i— j— l,j  — 1  +  tx$d-i,i—j,j-2- 
Using  the  recursion  relation  (10),  we  find  that 

A<id](x'  y)  =  y)  +  sA\- 7,V(*»  y) +  M!-Tj-x(x*  y) 

for  i  =  0, 1, . . . ,  d  and  j  =  0, 1, . . . ,  i,  provided  that  we  start  with  .4^0  =  r 2,  .4j2q  = 

2rIsI,  A^l  =  2 rztz,  A^l  =  sx,  A^\  =  2sztx,  and  A^2  =  The  polynomials  pXJ 
and  pyy  have  similar  expansions  in  terms  of  functions  satisfying  the  same  recursions, 
but  with  different  starting  values. 

Using  the  above,  we  get 

[Pzi]2  =  d2(d  -  !)2  bi^+j+ibU^+l+\exdli,j,k,l' 


i=0  >=0  k=0  1=0 


where 


eizdz,i,j,k/x^y)  :=  A\dj(x,y)  ■  Aid)i(xiy)- 

Let  e^Zy,i,j,k,i  eydJ,i,j,k,t  Le  the  relevant  expressions  involved  in  the  expan¬ 
sions  of  \pZy}2  and  \pyy]2,  and  let  e\dJk  l  :=  ^  ,  +  2e^),  j  t  |  +  Then 

it  follows  that  *,i  satisfies  the  stated  recursion  relation,  and 

[Dlt  -I-  2D2zy  f  D]y\p  =  d2(d  -  l)2  US  b,^+j+ib±^+i+ie(idjXi- 

i=0  ;=0  k= 0  /=0 

The  result  follows  since  the  entries  of  the  energy  matrix  are  obtained  by  inte¬ 
grating  the  expressions  c\jkl  over  the  triangle  T.  ■ 
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4.  The  cubic  and  quartic  cases 

In  this  section  we  consider  cubic  and  quartic  splines  as  they  are  the  most  likely 
to  be  useful  in  practice.  In  particular,  we  present  the  explicit  entries  in  the  energy 
matrices  E ^  and  E^.  Before  proceeding,  we  observe  that  by  the  symmetries 
involved,  in  describing  E^d\  it  is  not  necessary  to  give  all  of  the  (d  +  l)2(d  +  2)2/4 
entries.  First,  by  symmetry  it  suffices  to  work  only  with  the  upper-triangular  part 
of  the  matrix.  Moreover,  it  is  clear  that  in  the  upper- triangular  part,  whenever 
a  formula  occurs  involving  R,  5,  there  Eire  always  two  others  of  the  same  form 
involving  S,T  and  R,T  (cf.  the  E^K  £33*,  and  E$  entries  of  the  matrix 
given  above).  In  addition,  whenever  a  formula  appears  involving  either  one  or 
three  of  the  letters,  then  there  are  always  two  other  similar  formulae  obtained  by 
cyclic  permutation  (cf.  the  E[\\  E[W  and  E$,  or  the  Ei2\  E^l\  and  E^J  entries 
of  £<2>). 

The  following  theorem  computes  the  number  of  formulae  which  must  be  given 
in  order  to  completely  specify  the  matrix  E^. 

Theorem  2.  The  number  of  formulae  needed  to  specify  the  matrix  E(d^  is  q^, 
where  q-2  =  q-x  —  0,  q0  =  1,  and  where  for  k  >  0, 

*72  fc  =  2  k3  +  4k  +  q2k-3 
q2k-i  =  2  k3  —  3  k2  -f-  5  k  —  2  +  q2k-*- 

Proof:  The  result  follows  for  d  =  2  by  inspection  of  the  matrix  E^2’  given  above. 
Now  we  can  proceed  by  induction.  We  consider  the  recursion  for  g2*-i  first.  In 
the  first  row  it  is  easy  to  see  that  it  suffices  to  give  2  +  4  +  •  •  •  +  2Jfe  entries  -  this 
corresponds  to  the  products  of  6j  by  the  b{  associated  with  domain  points  in  the 
left  half  of  the  triangle  T.  For  example,  in  the  cubic  case  (cf.  Figure  2),  we  need 
the  (1, 1),  (1, 2),  (1,4),  (1,5),  (1,7),  (1, 8)  entries.  Next,  for  each  i  =  1, . . . ,  k  —  1,  in 
the  +  1-st  row  of  the  matrix  we  need  (n+1Kn+2l  _  3(21  —  1)  —  1  formulae.  In 

the  cubic  case  in  Figure  2,  these  are  the  (2, 2),  (2, 3),  (2, 4),  (2.5),  (2, 6),  (2, 9)  entries. 
This  accounts  for  all  rows  corresponding  to  coefficients  on  the  boundary  of  T.  To 
complete  the  proof  we  simply  apply  the  induction  hypothesis  to  the  coefficients 
lying  inside  the  triangle.  In  the  cubic  case  this  is  just  the  (5, 5)  entry. 

The  proof  of  the  recursion  for  q2k  is  similar.  In  this  case,  in  the  first  row  it 
suffices  to  give  1  -f  3  +  •  •  •  +  (2k  4-  1)  entries  -  this  corresponds  to  the  products  in 
the  left  half  of  the  triangle  T.  Next,  for  each  i  =  1, . . . ,  Jfc  —  1,  in  the  +  1-st 

row  of  the  matrix  we  again  need  ^-1Kn+7)  _  3(21  —  1)  —  1  formulae.  In  this  case  we 
also  need  to  do  the  ^±11  +  row?  which  requires  A:(/:  —  1)4-2  formulae.  Finally, 
we  apply  the  induction  hypothesis  to  the  coefficients  lying  inside  the  triangle.  ■ 

Using  Theorem  2  we  see  that  for  the  cubic  case  we  need  13  formulae,  while  for 
the  quartic  case  we  need  26  formulae. 


Figure  2.  A  typical  triangle  with  associated  coefficients 


Example  3.  The  thirteen  essential  formulae  for  the  entries  of  G  =  5 E^/9ac  are 

G<*{  =(RR)J 

G<*>  =2 RRRS  +  .5(RR  )* 

G™  MRS)1  +  RRRS 

G[*l  =2  RS  RT  +  RRRT  +  RR  RS 

G(*>  =.5  (RS)1 

g\*1  =  RSRT  +  .5  (RS)1 

G(,*’  -2 RRSS  +  2(RS)3  +  2RR  RS  +  (RR)1 

G(*l  =2  RRST  +  2  RS  RT  +  RR RT  +  RRRS  +  .5(RR)S 

G[*l  -2 SSRS  +  RRSS  +  1.5 (RS)1  +  2 RRRS 

G,*g  =2SS  RT  +  2RS  ST  +  RR  ST  +  2RS  RT  +  RR  SS  +  (RS  )J  +  2RR  RT  +  RR  RS 
G(,*J  =2 RTST  +  .5( RT)1  +  RRST  +  RS  RT  +  RRRT 
G(,*J  =RT  ST  +SSRT  +  RSST  +  (RT)1  +  RS  RT 

G(*l  =2  (SSTT  +  (ST)*  +  TTRS  +  RTST  +  SSRT  +  RSST  +  RRTT  +  (RT)’ 

+  RRST  +  RSRT  +  RRSS  +  (RS)*) 


Example  4.  The  26  essential  formulae  for  the  entries  of  H  =  5E^/4ac  axe 


H[*l  =6  (HR)’ 

H[*l  =  12RRRS  +  3  (RR)1 
H[*l  =6(RS)3 +6RRRS  +  (RR)J 
H[*l  =12  RSRT  +6RRRT  +6RRRS  +  (RR)1 
=3(RS)*  +  2RRRS 

=6RSRT  +  3(RS)2  +  2RRRT  +2 RR  RS 
H[%  =(RS)2 
H[*l,  =2RS  RT  +  (RS)1 
H[*lt  =(RT)J  +  2  RSRT  +  (RS)1 
H(*l  =12RRSS  +  12  (RS)1  +  12  RRRS  +4  (RR)1 
H^l  =12RRST  +12  RSRT  +6  RRRT  +6  RRRS  +2  (RR)1 
=12 SSRS  +  6 RRSS  +  9(RS)J  +  10RR  RS  +3 (RR)1 
H(3*l  =12  SSRT  +  12  RSST  +6  RRST  +  12  RSRT  +  6RRSS  +  6(RS)s  +  8  RRRT  +6  RRRS 
+  2  (RR)1 

H(*l  =12  RTST  +  3(RT)2  +  2RRRS  +  6RRST  +6RSRT  +4  RRRT  +  (RR)1 
H{*\  =6  SSRS  +6(RS)2  +  2RRSS  +6  RRRS 

=6 SSRT  +6 RSST  +  6SS  RS  +  10 RSRT  +  2RRST  +  2RRSS  +  4(RS  )2  +  6RR  RT 
+  4  RR  RS 

=6 RTST  +6 SS  RT  +6 RSST  +  4(RT)3  +  2 RRST  +  6 RSRT  +  2 RRSS  +  2 (RS)1 
+  4  RRRT  +2  RRRS 

H(3} o  =6 RTST  +  2(RT)7  +  2RRST  +  2RS  RT  +  2RRRT 
Hj*la  =2  RTST  +2  SSRT  +  2RSST  +  2SS  RS  +  3(RT  )*  +  4RS  RT  +(RS)J 
=2  RTST  +2SS  RT  +2RSST  +  2(RT)S  +  2R5  RT 
H**<  =«(SS)l  +  12SSRS  +  8  RRSS  +  10(RS)3  +  12RRRS  +  6(RR)3 

=1255  ST  +8  SS  RT  +  12  RS  ST  +#  SSRS  +  8  RRST  +  10RSRT  +4RRSS  +  5(RS)3 
+  6  RRRT  +6  RRRS  +  3(RR)3 

=8(ST)3  +  8RTST  +  6RSST  +  (RT)3  +  4  RRST  +  4RSRT  +(RS)3  +  2  RRRT 
+  2  RRRS  +  (RR)3 

=3(ST)3  +  8SSST  +8  RTST  +4  SSRT  +6RSST  +2  SSRS  +  3(RT)3  +  4RR  ST 
+  8RSRT  +2RRSS  +  2(RS)3  +  6RR  RT  +2  RRRS 
=12  SSTT  +  12(ST  )*  +  12TTRS  +  12RTST  +  12SSRT  +  12RSST  +  8RRTT  +  8  (RT)3 
+  8  RRST  +  12  RSRT  +  8  RRSS  +  8(RS)3  +  8  RRRT  +  8  RRRS  +  4  (RR)* 

=8  SSTT  +  8(ST)3  +  8SSST  +  10TTRS  +  10  RTST  +6SSRT  +  10RSST  +8SSRS 
+  6RRTT  +  8(RT)3  +  8  RRST  +  10RSRT  +4RRSS  +  8(RS)3  +  8RR  RT  +8  RRRS 
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Abstract  A  radial  basis  function  approximation  s(-)  from  7Zn  to  71  depends  on  a 
fixed  set  of  points  {x,}  in  7Zn  and  on  a  fixed  function  <j>(-)  from  7Z+  to  71,  as  it  has 
the  form  {s(x)  =  X^  A,  <t>(\\x  —  art- 1|2)  |  x^TV'}.  We  consider  the  case  when  {xj  is  the 
infinite  lattice  Zn  and  the  coefficients  {A,}  give  interpolation  on  the  lattice.  There¬ 
fore  we  study  the  cardinal  function  C(-),  which  is  in  the  space  of  approximations 
and  satisfies  {C(l)  —  S0e  |  (  €  Zn }.  By  employing  Fourier  transforms,  we  identify 
the  rate  of  decay  of  lC(x) |  to  zero  as  ||x||  — »  oo  for  several  useful  choices  of  <f>{-). 
Further,  an  algorithm  is  presented  for  calculating  C(x)  for  moderate  values  of  ||x||. 
It  provides  some  tabulations  of  the  cardinal  function  that  quantify  the  qualitative 
asymptotic  theory.  The  results  may  be  highly  important  to  the  development  of  new 
algorithms  for  multivariable  interpolation,  because  they  include  some  cases  where 
the  asymptotic  decay  to  zero  is  very  fast. 

Keywords  Cardinal  functions.  Fourier  transforms.  Gauss-Seidel  iteration. 
Interpolation.  Multivariable  approximation.  Radial  basis  functions. 


1.  Introduction 

Radial  basis  functions  provide  a  versatile  family  of  functions  from  7Zn  to  7Z  that  is 
highly  promising  for  the  approximation  of  functions  of  several  variables  in  computer 
calculations.  The  generic  radial  basis  function,  4>{-)  say,  is  from  7Z+  to  71,  and  a 
typical  radial  basis  function  approximation  with  n  variables  has  the  form 

s(x)  =  £A.XI|x-x,||2),  x€ft",  (1) 

i=i 

where  {A<  j  i  =  1,2,. ..,m}  and  {x,  |  i  =  1,2,. ..,m}  are  real  parameters  and  fixed 
points  in  7ln  respectively.  When  interpolating  to  values  of  a  function  /(•)  :  7Zn  —*7Z, 


it  is  usual  for  the  {x,  |  i  =  1,2, to  be  the  points  at  which  the  function 
values  are  given  and  for  the  parameters  {A;  |  i  =  1,2, . . . ,  m}  to  be  defined  by  the 
interpolation  conditions 

s{xi)  =  f(xi),  i  =  1,2, . . . ,  m.  (2) 

Of  course  the  points  {x;  |  i  =  1,2, should  ail  be  different,  and  then  for 
many  useful  choices  of  <p(-)  the  interpolation  conditions  are  guaranteed  to  define  the 
parameters  uniquely  (Micchelli,  1986).  This  important  work  was  reviewed  at  the 
previous  Shrivenham  conference  (Powell,  1987). 

An  obvious  advantage  of  expression  (1)  over  the  methods  that  are  employed 
usually  for  multivariable  approximation  is  that  this  expression  does  not  become 
more  elaborate  as  the  number  of  variables  is  increased.  When  m  is  large  and  4>(-) 
does  not  have  finite  support,  however,  then  the  work  of  calculating  s(x)  for  any  x 
can  be  much  greater  than  the  work  of  calculating  a  typical  piecewise  polynomial 
approximation.  The  purpose  of  the  present  paper  is  to  consider  another  question 
that  is  highly  important  to  the  usefulness  of  radial  basis  function  approximation, 
namely  the  localization  properties  of  interpolation.  In  other  words,  assuming  that 
s(-)  can  be  expressed  in  the  form 

s(*)  =  Y,n*i)Ci(x),  xenn,  (3) 

i=  1 

where  the  cardinal  functions  {(?,(•)  j  t  =  1,2,  ...,m}  are  independent  of  /(•)  and 
satisfy  Ci(xj)  =  Sij  in  order  that  the  interpolation  conditions  (2)  hold,  we  ask  how 
quickly  jC,(x)|  Lends  to  zero  as  ||x  —  x,||  becomes  large.  Here  ||  •  ||  denotes  any 
norm  on  lZn.  We  see  that  this  question  is  directly  relevant  to  the  effect  on  s(  )  of 
perturbations  to  the  given  function  values  {/(x,)  1 1  =  1,2, ...,m}. 

Perhaps  the  best  known  example  of  localization  properties  without  finite  support 
is  cubic  spline  interpolation  to  values  of  a  function  of  one  variable  at  equally  spaced 
points  throughout  the  real  line.  In  this  case  the  only  zeros  of  C;(  )  occur  at  the  data 
points  {ij  |  j  ^t},  and  the  cardinal  function  satisfies  the  relation 

C,(x,±0)  =  (^3-2)  Ci(x,±[0 -/»]),  0>2h,  (4) 

where  h  is  the  spacing  between  data  points  (see  Powell,  1981,  for  instance).  Thus, 
if  k  is  the  number  of  data  points  between  x  and  x,,  then  |C(x)|  is  bounded  above 
by  a  multiple  of  (2  —  \/3)*.  This  example  can  be  construed  as  radial  basis  function 
approximation  with  n  =  1  and  {<£(r)  =  r3  j  re7£+}. 

We  restrict  attention  to  the  case  when  the  set  of  data  points  {x,}  is  the  infinite 
lattice  Zn  of  points  in  1Zn  with  integer  components,  in  order  that  several  decay 
properties  of  cardinal  functions  can  be  derived  by  Fourier  techniques.  It  should  be 
noted,  however,  that  some  of  the  main  advantages  of  radial  basis  function  interpo¬ 
lation  occur  when  the  data  points  are  in  general  position.  We  let  (  be  the  generic 


v*v 


a 


« 


I 


< 


point  of  2",  and,  using  translational  symmetry,  we  write  equation  (3)  in  the  form 


3(*)=  £  MC(x-e),  xen\  (5) 

t£Zn 


where  now  there  is  a  single  cardinal  function  that  satisfies  {C(j)  =  60}  |  j£Zn}. 

Provided  that  all  the  sums  and  integrals  in  this  paragraph  are  uniformly  conver¬ 
gent,  it  is  suitable  to  let  C(-)  be  the  function 


c(*)=  £  <*tf||x-£||a),  X€H* 
eezn 


(6) 


where  the  coefficients  have  the  values 


ct 


=  -J_  / 

(2jr)n  Jr- 


Jt.B 


(2 x)-  ||2) 


d0, 


(7) 


Here  i  is  \/—  1  and  £.5  denotes  the  scalar  product  between  £  £  Zn  and  0  6  'Rn .  We 
see  that  C(-)  is  in  the  correct  linear  space,  and  that  the  identity 


cu)  =  £  c<m  -  £iw 

tez" 

=  £ 
tezn 

_ 1_  /  E,».  e'»-»^(|K||,) 

(2»)"  y,-,,r 

-  *“'*■*'•  je*"'  (8) 

is  satisfied,  which  makes  expression  (6)  plausible.  This  way  of  identifying  the 
cardinal  function  is  successful  when  (</>(r)  =  e~r 2  \  r  €  7£+},  for  example,  but, 
due  to  the  conditions  of  absolute  convergence,  it  cannot  be  applied  directly  unless 
E/e-Z"  Wll*ll*)l  finite.  Therefore  we  employ  Fourier  methods  to  study  the  cardinal 
functions  of  the  choices 


4>{r)  -  r 

4>{r)  =  r3 _ 

4>(r)  =  \Jr2  -f  c2 
<£(r)  =  1  /  Vr2  c 2 

$(r)  =  r2  log  r 


(linear) 

(cubic) 

(multiquadric)  . 

(inverse  multiquadric) 
(thin  plate  spline) 


(9) 


We  are  interested  in  these  radial  basis  functions  because  each  one  can  give  s  =  f 
when  /(•)  is  a  low  order  polynomial  (Jackson,  1987;  Buhmann,  1988b).  Specifically, 
letting  II?  be  the  space  of  all  polynomials  from  TV1  to  H  of  total  degree  at  most  q,  the 
interpolation  formula  (5)  reproduces  each  /(•)€  II”,  where  q  =  n,  n  +2,  n,  n  - 2  and 
n  +  1  in  the  linear,  cubic,  multiquadric,  inverse  multiquadric  (assuming  n  >  2)  and 


« 


thin  plate  spline  cases  respectively.  In  each  of  these  cases  C(-)  has  the  form  (6)  for 
certain  coefficients  {c(  |  Zn]  that  make  the  sum  (6)  absolutely  convergent,  so  the 
cardinal  function  is  well-defined.  Further,  the  sum  (5)  is  also  absolutely  convergent 
for  every  x  £  TV  and  every  /(•)  £  II",  and  it  gives  {s(x)  =  /(x)  |  x  €  TV1}  when 
/(•)GlIJ.  It  should  be  noted,  however,  that  the  double  sum 

E/M  do) 

tezn  jezn 

which  is  obtained  by  substituting  expression  (6)  in  formula  (5),  need  not  be  abso¬ 
lutely  convergent.  Indeed,  if  we  were  allowed  to  rearrange  terms,  then  the  factor 
YLteZ"  f{f)  ||x  —  i — j'  l 1 2 )  would  multiply  c:,  but  when  /  =  1  this  factor  is  a  divergent 
sum  for  all  of  the  choices  (9). 

For  example,  consider  the  case  when  n  =  1  and  {<f>(r)  =  r  |  r  6  TV'}.  The  co¬ 
efficients  of  the  cardinal  function  (6)  have  the  values  Cq  =  —  1,  c_i  =  C!  =  ~  and 
{c*  =  0  |  |^|  >  2}  in  order  that  C(-)  is  the  hat  function  of  piecewise  linear  interpo¬ 
lation.  Hence  formula  (5)  reproduces  all  linear  polynomials.  Thus  the  space  that 
is  spanned  by  the  radial  basis  functions  {  {||x  —  £||2  |  x  £  TV)  |  (.£  Z]  includes  n}, 
although  no  nonzero  linear  polynomial  can  be  written  as  ||x  —  ^||2  |  x£7£}. 

Therefore,  when  studying  the  cases  (9)  we  take  the  view  that  the  approximation  s(-) 
is  to  be  in  the  linear  space  that  is  spanned  by  functions  of  the  form  (6),  which  admits 
the  very  powerful  observation  in  the  previous  paragraph  that  q  increases  with  n.  Of 
course  this  point  of  view  is  unnecessary  when  m  is  finite  in  expression  (1).  It  is  also 
unnecessary  when  {x,}  =  Zn  and  {^(r)  =  e~rJ  |  r£ TV]  for  example,  but  in  this  case 
the  interpolation  formula  (5)  does  not  give  s  =  /  when  /  =  1  (Buhmann,  1988b). 
Therefore  in  practice  the  Gaussian  radial  basis  function  is  usually  far  less  successful 
than  the  nonin tegrable  functions  (9). 

We  study  the  cardinal  function  C(-)  theoretically  in  Section  2.  Here  we  recall 
from  Buhmann  (1988b)  an  explicit  expression  for  the  Fourier  transform  of  C(  )  that 
is  valid  for  all  of  the  choices  (9),  and  we  also  recall  that  the  asymptotic  decay  rate 
of  |C(x)|  to  zero  as  ||x||  — ►  oo  can  be  deduced  from  the  differentiability  properties 
of  this  transform.  This  analysis  identifies  the  dominant  term  of  C(x)  for  large  ||x|| 
when  <f>(r)  =  r  and  n  =  2. 

In  Section  3  we  present  a  procedure  for  calculating  values  of  C(x).  It  employs  a 
quasi-interpolating  function 

V>(*)  =  £  7^(11* -*||2),  xenn,  (11) 

teZ 

where  Z  is  a  finite  subset  of  Zn ,  that  ideally  possesses  the  diagonal  dominance 
condition 

10(0)1  >  E'WOI-  (12) 

l€Zn 


I 


Here  and  throughout  this  paper  the  prime  on  the  summation  indicates  that  the  l  =  0 
term  is  omitted.  This  function  is  used  in  an  iteration  of  Gauss-Seidel  type  to  obtain 
an  approximation  C<,(-)  to  C(-)  that  satisfies  the  cardinality  equations 

C„(t)  =  60t,  iez\  IKIU  <6,  (13) 

for  some  fixed  integer  b.  We  find  that  this  approximation  is  sufficiently  accurate  to 
give  good  estimates  of  the  true  cardinal  function  for  ||x||oo  <  \b. 

Thus  several  tables  and  graphs  were  calculated  that  show  properties  of  C(x)  for 
n  =  2  and  n  =  3.  We  find  when  n  =  3  and  <£(r)  =  r,  for  example,  that  {\C{x)\  <  10~6  | 
llxlloo  >  5},  which  is  superior  to  the  decay  rate  (4)  of  cubic  spline  interpolation. 
These  numerical  results  are  given  and  discussed  in  Section  4,  and  we  note  that  they 
suggest  several  topics  for  further  research. 


2.  The  Fourier  transform  and  decay  properties  of  the  cardinal  function 

The  Fourier  transform  of  a  continuous  and  absolutely  integrable  function  {g(x)  | 
x  G  7Zn }  is  defined  by  the  formula 

g(t)=f  e~'xtg(x)dx ,  t£H\  (14) 

JHn 


and,  when  g(-)  is  continuous  and  absolutely  integrable  too,  the  inverse  formula  is 
the  relation 

</(*)  =  7 tzzL  e,xt9(t)dt,  xenn,  (is) 

(2*)"  Jnn 


(see  Stein  and  Weiss,  1971,  for  instance).  In  this  section  we  derive  some  decay 
properties  of  the  cardinal  function  C(-)  from  its  Fourier  transform  C()  for  each  of 
the  radial  basis  functions  (9). 

It  is  proved  by  Buhmann  (1988b)  that  in  all  these  cases  C(-)  has  the  remarkably 
simple  form 


C(t)  = 


2*fc||2)’ 


t  g  nn, 


(16) 


where  {^(||t||2)  I  is  the  generalized  Fourier  transform  of  the  radially  sym¬ 
metric  function  {<^(]|x||2)  |  Jones  (1982)  presents  an  excellent  introduction 

to  generalized  transforms,  and  shows  that  the  functions 


^(r)  =  r-”"1 
^(r)  =  r~n~3 

4>{r)  =  (c/r)<n+1>/2Ar(n+1)/,(cr) 
4>(r)  =  (c/r)("-»/2A(n_1)/2(cr) 

Hr)  =  r-n"2 


(linear) 

(cubic) 

(multiquadric) 

(inverse  multiquadric) 
(thin  plate  spline) 


(17) 


i 


are  appropriate  to  the  radial  basis  functions  (9),  except  that  for  simplicity  we  have 
suppressed  some  constant  normalization  factors  because  they  are  irrelevant  to  the 
ratio  (16).  Here  Kv{  )  is  the  t/-th  modified  Bessel  function  as  defined  in  Abramowitz 
and  Stegun  (1970). 

We  see  that  in  all  these  cases  the  generalized  Fourier  transform  {0(||*||2)  I  *  €7£n} 
is  unbounded  at  the  origin,  but  otherwise  it  is  positive,  continuous  and  absolutely 
integrable  over  every  closed  part  of  'R.n  that  excludes  the  origin.  In  view  of  equations 
(16)  and  (17),  this  unboundedness  implies  the  values  C(0)  =  1  and  {C(2nk)  =  0  | 
k  €  2n\{0}  },  which  are  important  to  the  fact  that  interpolation  reproduces  low 
order  polynomials  (Jackson,  1987).  Further,  when  t  is  not  in  the  set  of  lattice  points 
{2irjfc  |  k  6  ■£"},  the  denominator  of  expression  (16)  is  an  absolutely  convergent  sum 
that  is  periodic  and  bounded  away  from  zero.  It  follows  from  all  these  remarks  that 
C(-)  is  well  defined  when  we  substitute  g(-)  =  C(-)  in  the  inverse  Fourier  transform 
formula  (15). 

Buhmann  (1988b)  uses  some  properties  of  generalized  functions  to  prove  that  this 
C(-)  is  the  required  cardinal  function,  but  the  following  simple  analysis  is  sufficient 
when  certain  sums  and  integrals  are  absolutely  convergent,  including  the  conditions 
that  {<^(||a:||2)  |  x€7Jn}  is  the  classical  inverse  Fourier  transform  of  {0(||*||2)  I  *G7£n} 
and  YlkeZ"  0(11*  —  2tt Ar|j2)  is  nonzero  for  all  t.  For  example,  these  conditions  hold  for 
the  Gaussian  radial  basis  function  {<£(r)  =  e~r2  |  rg7J+}  because  {^(||i||2)  |  te7£"} 
is  a  constant  multiple  of  {e'll'lb/4  |  t  €  7ln}.  Corresponding  to  the  identity  (8), 
equations  (15)  and  (16)  give  the  cardinality  conditions 


c«  -  tsfL 


'0(11*112) 


(2*)nJK’'Zk€Zn4>(\\t-2xk\\2) 

=  _J_  f  E^ne^-^^llt-2^112) 

(2tt)"  /[-»., )»  *(||t  -  27r*||2) 

-  WrL,r  c“'dt  -**’  ’*zn’  (18) 

where  the  middle  line  depends  on  the  periodicity  of  the  denominator  and  the  last 
line  on  e*-^2*^  =  1.  Moreover,  to  show  that  C(-)  has  the  form  (6),  we  note  that  the 
continuous  periodic  function 

U*<)  =  1/  £  0(ll*-27rfc||2),  teKn,  (19) 

k£Zn 

is  equal  to  its  Fourier  expansion,  so  we  have  the  identity 

u >(<)=£  (20) 

<6  Zn 

where  the  coefficients  have  the  values 


Thus  C(-)  is  the  function 
C(x)  = 


-J-/ 

(2x)n  A 

— L_  / 

(2jr)n  A 


C(f)dt 


£“<(5^^  *«•>*> 

£  Q#ll*-*ll2),  *eftn, 

*62" 


(22) 


where  the  third  and  fourth  lines  depend  on  equations  (20)  and  (15)  respectively. 
This  expression  is  identical  to  equation  (6)  as  required.  Further,  it  can  be  deduced 
from  the  Poisson  summation  formula  (see  Stein  and  Weiss,  1971,  for  instance)  that 
the  coefficients  (7)  and  (21)  are  the  same. 

Next  we  consider  the  localization  properties  of  interpolation  that  can  be  deduced 
from  C(-).  If  {C(t)  |  t  6  7£n }  has  absolutely  integrable  second  derivatives  and  is 
sufficiently  smooth  for  large  ||t||,  then  integration  by  parts  applied  to  formula  (15) 
gives  the  relation 


C(x)  = 


— V  /  eix  tc(t)dt 
(2tt)»  H*  v  ' 

_ - - / 

(27r)n(ixi)  A 


Tln  at  i 


which  implies  the  identity 

IMIjCM  = 


—  / 

(27r)n  J-Rn 


e,xt  [v2c(t)] dt,  ier, 


(23) 


(24) 


and  there  is  an  upper  bound  on  the  modulus  of  the  right  hand  side  that  is  inde¬ 
pendent  of  x.  Similarly,  if  C(-)  has  absolutely  integrable  derivatives  of  order  j  and 
enough  smoothness  at  infinity,  this  technique  provides  the  inequality 


\C(x)\  <  Aj  [1  +  ||x||3] 


-J 


xeizn. 


(25) 


for  some  positive  constant  A  j.  Thus  the  decay  properties  of  the  cardinal  function 
C(  )  are  intimately  related  to  the  differentiability  of  its  Fourier  transform  C(  ). 

Therefore,  as  in  Buhmann  (1988b),  we  consider  the  differentiability  of  the  func¬ 
tion  (16)  for  each  of  the  choices  (17)  of  <£(•).  In  the  linear,  thin  plate  spline  and 
cubic  cases  we  have  the  Fourier  transform 

<5(o  =  1 1 + wr”  e  'ii<  -  2»*nr-"  r , 

*62" 


tenn , 


(26) 


I 


where  m=  1, 2  and  3  respectively.  We  see  that  this  function  is  infinitely  differentiable 
throughout  7ln  when  n  +  m  is  even,  which  gives  the  important  conclusion  that  a 
bound  of  the  form  (25)  is  satisfied  for  every  positive  j.  In  fact  in  these  cases  jC(ar)| 
decays  exponentially  as  ||x||  -*•  oo  (Madych  and  Nelson,  1987;  Buhmann,  1988b), 
which  means  that  the  condition 


|C(x)|  <  AmIWI,  (27) 

is  satisfied  for  some  constants  A  >  0  and  fi<  1.  One  purpose  of  the  numerical  results 
of  Section  4  is  to  indicate  the  magnitude  of  fi  when  n  —  2  and  n  =  3. 

When  n+m  is  odd  in  expression  (26),  however,  then  C(-)  has  singularities  at  the 
grid  points  t€2xZn  that  restrict  the  value  of  j  in  inequality  (25).  The  singularities 
at  t= 0  and  when  t£{2Ttk  |  fc€-2"\{0}  }  are  shown  in  the  expansions 


c{t)  =  i  -  n<iirm  E'p^iir— + o(iic+m+1) 

k€2n 


(28) 


and 

C{t)  =  ||t  -  2*k\\’fm  pr/fcllj"-”1  +  0{\\t  -  2a-jfc||”+m+1)  (29) 

respectively,  and  it  is  the  contribution  from  these  singularities  to  {C(x)  |  i€H”) 
through  the  first  line  of  equation  (23)  that  dominates  the  magnitude  of  |C(x)|  for 
large  ||x||.  Instead  of  integrating  by  parts,  we  pick  out  this  contribution  from  the 
fact  that  the  generalized  inverse  transform  of  {||tj|2+m  |  1 6  77"}  when  n  +  m  is  odd 
is  the  function 

2"+“  f(n  +  Jm) 


x£nn. 


(30) 


Specifically,  the  terms  from  the  singularities  of  expressions  (28)  and  (29)  are  given 
explicitly  in  the  equation 


CM  “ 


(2x) 


*62" 


+  £  'll t  -  2irk\\^m  ||27r*||r-m  +  •  •  •  ]  dt 

k£Zn 

(SF  L  e“' ‘ [  -  w~”  +  •  •  ■  ] 

+  (  J2  'e,x(t+2rfc)  ||t|£+m  ||2x*||2-”-m  +  •  -  -}dt 

k€2n 

.  +  /  V'  'f  1  ,  Iriz.k]  |IOTi.||-n-ml  ||  ||-2n-« 

War(-ln-lm)  w  2  1111,2 

+o(||x||2-2n-), 


“f*  •  *  *  dt 


(31) 


which  is  valid  for  all  large  ||x||.  We  see  that  the  sum  in  the  last  line  of  this  expression 
is  absolutely  convergent,  real,  nonpositive  and  periodic  in  x  €  7Zn.  Further,  it 
vanishes  at  the  grid  points  x  6  2n  in  accordance  with  the  cardinality  conditions 

{C(£)  =  Mfezn}- 

Later  we  give  particular  attention  to  the  linear  radial  basis  function  <t>(r)  =  r 
when  n  =  2  and  m  =  1.  Therefore  we  note  that  in  this  case  expression  (30)  has  the 
value  9/(27r||x|j|),  and  for  any  x  it  is  easy  to  work  out  numerically  the  sum  in  the 
last  line  of  equation  (31).  Thus  we  find  the  relations 

C(x)»  -0.05747  ||xll2"5,  [x  +  (i,0)]€Z2  1 

C(x)«  —0.06734  ||x[|2-5,  [x  +  (i,I)]6  22  J  ’ 

for  the  values  of  the  cardinal  function  at  the  midpoints  of  edges  of  grid  squares  and 
at  the  midpoints  of  grid  squares  respectively.  These  asymptotic  results  agree  well 
with  the  calculations  of  Section  4  for  moderate  values  of  ||x||. 

The  following  argument  shows  that,  for  even  n,  the  asymptotic  decay  properties 
of  C(-)  in  the  multiquadric  and  inverse  multiquadric  cases  are  similar  to  expression 
(31).  The  dominant  singularity  of  {Kv(0)  |  067Z+}  at  the  origin  is  a  multiple  of  0~u , 
so  we  let  m  =  1  and  m  —  —  1  for  multiquadrics  and  inverse  multiquadrics  respectively, 
in  order  that  lines  3  and  4  of  expression  (17)  give  the  relation 

4{r)  =  Hm  r-n~m  +  0(r-""m+1),  r  G  7 *+,  (33) 

where  /im  is  a  constant.  It  follows  from  formula  (16)  that,  instead  of  equations  (28) 
and  (29),  we  now  have  the  expansions 

c(<)  =  i  -  Hiiir-  e  >(112**11,) + o(ii(iirm+i)  04) 

kezn 


C(t)  =  H~mx  II*  -  2^nrm  hP*k\U)  +  0(11*  -  27r*||r-",+1).  (35) 

Thus  expression  (31)  is  still  valid  except  that  the  sum  inside  the  braces  is  replaced 
by  the  term 


E'l-i  +  i 


k]0(l|2**||2). 


In  order  to  be  more  explicit,  we  recall  from  Abramowitz  and  Stegun  (1970)  that, 
when  v+l  is  a  positive  integer,  the  i/-th  modified  Bessel  function  is  the  finite  sum 

K„(0)  =  ^e-9^  0671+,  (37) 

7=0  \v  9  2  /• 

which  after  some  manipulation  gives  the  value 

-  iio-LM-n-m  -||2^*||j  {±[n  +  m- 1] )!  (n +m - q - 1 )!  ||4tc^||^ 

"  1,2  (J[n+m-l]-,)!(n+m-l)!,!  ‘ 


When  this  expression  replaces  ||2rA:||jn-m  in  equation  (31),  we  have  the  dominant 
asymptotic  term  of  the  cardinal  function  C()  for  the  multiquadric  (m  =  1)  and 
inverse  multiquadric  (m  =  — 1)  radial  basis  functions,  provided  that  n  is  even. 

In  particular,  taking  n  =  2,  m  =  l,  c  =  |  and  x  at  the  midpoint  of  an  edge  of  a 
grid  square,  this  analysis  gives  the  relation 


C(x)  =  + 


Wlli  112**11; 


r]e-c,l^||x||2-s  +  0(||xj|2-5) 


=  -0.005212  ||x||2-5  +  o(||x||2-5),  [x  +  (1  0)]eZ2, 


when  ||x||  is  large.  One  unexpected  conclusion  from  equations  (32)  and  (39)  is  that 
in  two  dimensions  multiquadric  interpolation  can  have  better  localization  properties 
than  linear  interpolation  (<p(r)  =  r). 

If  n  is  odd,  however,  then  neither  multiquadrics  nor  inverse  multiquadrics  give 
the  exponential  decay  (27).  In  these  cases  n  +  m  is  even  in  equation  (33),  so  the 
r-n-m  term  does  not  cause  a  singularity  in  the  Fourier  transform  (16)  of  the  cardinal 
function,  but  there  is  some  loss  of  differentiability  due  to  the  expansion  of  the 
modified  Bessel  function  {/t(n+m)/2(cr)  j  r€"£+}  at  r  =  0.  Specifically,  the  dominant 
term  of  4>(r)  for  small  r  and  the  first  term  that  gives  a  singularity  are  nmr~n~m  and 

log  r  respectively,  where  and  /km  are  constants.  Now  equation  (16)  implies 
the  relation 


C(f)«l-(^(||<l|2)]-1i:V(||2Jr*||2) 

fce2" 

iimr-ii +•••+— iKiir™  i°g  + ■  r1  e  <«> 

Mm  k£Zn 


for  small  t.  Thus  the  leading  singularity  in  C(-)  is  proportional  to  {||f||2n+2m  log  ||t||2  | 
t  €  7£n } .  The  generalized  inverse  Fourier  transform  of  this  expression  is  a  multiple  of 
{||x||23n-2m  |  x£lZn}.  Hence,  for  odd  n  and  large  ||x||,  |C(x ) |  is  0(||x||23n_2)  and 
0(||x||23n+2)  for  the  multiquadric  and  inverse  multiquadric  radial  basis  functions 
respectively. 


3.  The  calculation  of  cardinal  functions 

The  results  of  Jackson  (1988)  suggest  that,  when  n  =  3  and  {<£(r)  =  r  |  r£"7l+},  it  is 
possible  for  a  quasi-interpolating  function 

Mx)  =  £7<0(l!*-*l!a)»  ^672", 

tzt 


(41) 


where  2  is  a  finite  subset  of  Zn  and  {7*  1 i 6  2}  is  a  set  of  constant  coefficients,  to 
possess  the  diagonal  dominance  condition 

W«)l>  I'lffll-  (42) 

tezn 

In  this  case,  with  the  normalization  ^(0)  =  1,  one  can  deduce  from  the  Gauss-Seidel 
iteration 

ct*+1i(x)  =  cW(x)  -  {C-W(j)  -  S0j}  Hx  -  j),  xen\  (43) 

j  being  a  vector  from  Zn  that  depends  on  k,  that  a  cardinal  function  of  interpolation 
exists.  One  uses  this  formula  to  generate  a  convergent  sequence  of  approximating 
functions  {(7l*+1l(-)  j  k  =  1,2,3,...}  whose  limit  satisfies  {Cl°°l(£)  =  Sot  I  t€Zn}. 
This  technique  is  mentioned  because  in  this  section  we  present  a  procedure  for  the 
calculation  of  approximations  to  cardinal  functions  that  includes  an  iteration  of 
Gauss-Seidel  type  that  is  derived  from  equation  (43). 

Our  procedure  begins  by  generating  a  quasi-interpolating  function  of  the  form 
(41)  in  a  way  that  is  described  later.  In  order  to  take  advantage  of  symmetry,  we 
let  2  =  2 "  for  some  small  integer  q  (usually  q  =  3  or  q  —  4),  where  we  define  2"  to 
be  the  set  2nn[— q,  q]n.  Thus  the  coefficients  of  expression  (41)  are  chosen  to  satisfy 
7pi  =  7*,  where  Pi  is  any  vector  in  Zn  whose  components  have  moduli  that  are  a 
permutation  of  the  moduli  of  the  components  of  i.  In  this  case  we  say  that  Pi  and 
i  are  equivalent.  Each  function  C^(-)  has  the  form 

cM(x)=  ^^(x-*),  (44) 

a  typical  value  of  the  integer  6  being  6=20,  and  each  integer  vector  that  is  analogous 
to  j  in  equation  (43)  is  restricted  to  the  set  2".  Therefore  in  the  limit  k—*  oo  we 
try  to  achieve  the  conditions 


cM(f)  =  «W,  iez?,  (45) 

which  correspond  to  the  equations  (13)  with  Cfc(  )  =  C^°°'(  ).  We  see  that  expressions 
(44)  and  (45)  give  a  (26+  l)nx(26+  l)n  system  of  linear  equations  in  the  coefficients 
I  f€2"}  that  is  to  be  solved  by  a  Gauss-Seidel  iteration.  We  take  advantage 
of  the  equivalence  symmetry  to  reduce  the  number  of  unknowns  to 

^(6+  1)(6  +  2)  and  |(6  +  1)(6  +  2)(6  +  3)  when  n  =  2  and  n  =  3  respectively,  which 
are  the  only  values  of  n  in  the  numerical  results  of  Section  4. 

To  begin  the  iteration  we  set  C^(-)  =  0(-)  and  k  =  1.  For  each  k  the  vector 
2"  is  chosen  to  maximize  the  residual  {fC^C?) —  ^>>1  I  j€  Z?}-  If  this  maximum 
residual  is  less  than  a  preset  tolerance  the  calculation  ends,  rind  from  now  on  we 
use  the  notation  C(,(-)  to  denote  the  final  C^(-).  Otherwise  a  step  of  Gauss-Seidel 
type  is  applied  to  give  C^+1l(-),  but  equation  (43)  is  not  used  as  it  stands  because 


it  would  lose  the  equivalence  symmetry.  Instead  we  define  V(j)  to  be  the  subset  of 
Z"  that  is  composed  of  j  and  all  other  vectors  that  are  equivalent  to  j,  and  we  let 
C(fc+1l(-)  be  the  function 

Clk+1\x)  =  CM(x)-{CM(j)-6Q]}  £  0(x-m),  x€H\  (46) 

m€V(j) 

Because  the  symmetry  implies  {CM(m)  =  |  TnE'P(j)}  and  because  t/>(0)  =  1 , 

it  can  be  shown  that  this  iteration  gives  the  inequality 

£  |cIk+l,(*)-M  <  £  |cw(0-4*l 

+  £  |C«(m)-<U  £  l*<<  -">)!•  (4T) 

Therefore  convergence  is  guaranteed  if  condition  (42)  holds.  The  iterative  procedure 
terminates  quite  efficiently  in  all  of  the  calculations  that  are  mentioned  in  the  next 
section,  but  the  diagonal  dominance  (42)  is  not  obtained  in  some  of  these  cases. 

Several  of  these  calculations  require  more  than  24  hours  of  running  time  on  a  Sun 
3/50  workstation,  because  of  the  number  of  terms  that  occur.  For  example,  when 
n  =  3,  6  =  20  and  Z  —  Z%,  the  computation  of  C*(x)  for  general  x  requires  103,823 
different  values  of  4>{r)  to  be  determined  and  each  one  involves  a  square  root  or  a 
logarithm.  It  is  therefore  important  to  plan  the  details  of  the  algorithm  carefully. 
The  following  features  are  included  in  our  iterative  procedure. 

Before  beginning  the  iterations  we  calculate  the  values  {rp(l)  i  of  the 

quasi-interpolating  function  (41),  and  for  each  k  we  have  available  the  coefficients 
{fi^  |  i  6  2?},  the  residuals  (C^(£)  —  Sot  |  1 G  2£}  and  the  value  of  l  that  gives 
the  residual  of  maximum  modulus,  which  is  the  j  of  formula  (46).  In  view  of  this 
formula,  the  new  coefficients  I  ^  €  Z^\P(j)}  are  the  same  as  the  old  ones 

|  t€Z"\P(j)}  but  the  {/4*1  |  are  overwritten  by  {/4*1  |  ^V(j)} 

where  —  C^(j)  —  6oj.  Further,  the  residuals  are  overwritten  by  the  numbers 

(d*+1,(£)  -  M  =  {C<fc,(0  -  Sot}  -  Plk]  £  w-m),  (ez?,  (48) 

mCVti) 

and  during  this  process  the  next  value  of  j  is  determined.  Because  the  values  of 
V>(')  in  equation  (48)  are  available  explicitly,  there  are  no  calculations  of  radial  basis 
functions  during  the  iterations  and  the  amount  of  work  per  iteration  is  independent 
of  the  number  of  terms  in  Z  =  Z".  In  all  of  these  operations  we  take  advantage  of  the 
equivalence  symmetry.  The  computer  program  stores  and  updates  only  one  of  the 
coefficients  {/4**}  and  one  of  the  residuals  {CW(£)  -  <5or}  for  each  of  the  equivalence 
sets  V((),  but,  in  order  to  save  some  work  when  equation  (48)  is  applied,  it  stores 
i){l)  for  every  t  in  Znn[0,26]B. 


Another  feature  of  this  program  is  that  it  allows  a  range  of  values  of  b.  They  are 
treated  in  ascending  order,  and  we  set  CM(-)  =  t/>(-)  only  when  b  is  least.  For  each  b 
the  iterations  continue  until  the  termination  condition  gives  C(,(-),  and  this  approx¬ 
imation  to  the  cardinal  function  is  chosen  as  the  C^(-)  of  the  next  6.  This  feature 
is  useful  because  comparing  tabulations  of  Ct(-)  for  several  values  of  b  provides  a 
good  indication  of  the  accuracy  of  these  approximations  to  the  cardinal  function. 
An  example  of  such  a  comparison  is  considered  later. 

The  coefficients  {7*  |  f  6  2"}  of  the  quasi-interpolating  function  (41)  are  de¬ 
termined  by  a  linear  programming  procedure  that  tries  to  achieve  inequality  (42) 
subject  to  the  normalization  condition 

0(0)  =  I-  (49) 

Because  the  sum  (42)  is  infinite,  a  finite  integer  a  satisfying  a>q  has  to  be  chosen. 
Then  the  coefficients  are  calculated  to  minimize  the  objective  function 

T.  w»i  =  £  1  E  (50) 

iezs  jez:  te_z; 

Except  in  the  n  =  2  inverse  multiquadric  case,  condition  (49)  is  augmented  by  further 
linear  equality  constraints  that  help  |t/>(x)|  to  become  small  as  ||x||  — +00. 

Most  of  these  constraints  are  contained  in  the  statement  that,  except  in  the  n  —  2 
inverse  multiquadric  case,  ip(-)  shall  have  the  form 

0(*)  =  E  7<{E*?0(H*-*ll2)},  (51) 

where  6?  is  the  central  difference  operator 

62Mx-l\\2)  =  *(||x— e,-£||2)  -  2*(||x-*||2)  +  «H||x+e,-£||2),  (52) 

e,  being  the  unit  vector  along  the  i-th  coordinate  direction  of  1Zn.  In  addition  to 
helping  the  boundedness  of  0(  ),  expression  (51)  has  the  strong  advantage  in  the 
linear,  cubic  and  multiquadric  cases  that  some  cancellation  can  be  done  analytically 
when  tp(x)  is  computed  for  large  ||x||.  For  example,  the  identity 

{(x1-1)j+q2}s  -  2{x2  +  q2}j  -I-  {(Xi  +  l^  +  a2^ 

=  8  a3  /  ( [  {(x,  -  l)2+a2}*  {(x,  +  l)2  +  a2}*  +  x\  +  a2  -  1  ] 

x  ({(x,-l)2+a2}i  +  2{x2  +  a2}*  +  {(x1  +  l)2  +  a2}i  ) )  (53) 

is  highly  useful  for  large  X!  when  0(r)  =  r  or  y/(r 2  -f-c2).  Such  identities  improve  the 
accuracy  of  many  of  the  numerical  results  of  Section  4,  particularly  when  4>(r)  =  r 
and  n  =  3,  because  in  this  case  some  values  of  the  cardinal  function  are  given  whose 
moduli  are  less  than  10-l°. 


We  also  include  the  constraints 


£  V  =  £  7r(4+^  +  ^+42-6^!)  =  0,  (54) 

E  7<  =  E  7<  +  +  *?+*2+^  -  3  [#S+#3+#?])  =  o  (55) 


E  7/=  E  7/(^+^+^)=  E  W<i+4+*S-3[#5+#2+#!]) 

*€*»_,  tez;_t 

=  E  +#3+^+^  -  is  W!) 

=  E  7r(2(^+^  +  ^]'l-15[£^22+^+^_90£2£2f2)  =  0  (56) 


for  the  thin  plate  spline  when  n  =  2,  the  linear  and  multiquadric  functions  when 
n  =  3,  and  the  cubic  radial  basis  function  when  n  =  3  respectively.  We  see  that  the 
objective  function  (50)  and  all  the  constraints  of  each  linear  programming  problem 
provide  the  required  symmetry  {7p/  =  7r  |  PI  E  V(£)},  so  we  reduce  the  number  of 
variables  of  this  calculation  to  the  number  of  equivalence  sets  V(£)  in  Z”  or  in  Z"_lf 
the  choice  depending  on  whether  we  work  with  {7/}  or  with  {7*}.  We  set  <7  =  3  in 
the  n  =  2  inverse  multiquadric  case  and  9  =  4  otherwise.  In  all  cases  we  let  o  =  6  or 
a  =  4  for  n  =  2  or  n  =  3  respectively.  No  difficulties  occurred  in  the  computations  of 
the  quasi-interpolating  functions  x^(-)  by  some  software  that  was  developed  by  one 
of  the  authors  (MJDP)  for  the  solution  of  general,  linearly  constrained,  optimization 
problems. 

The  purpose  of  each  of  the  constraints  (54),  (55)  and  (56)  is  to  optimize  the 
asymptotic  decay  properties  of  t/>(-).  Specifically,  they  ensure  that  the  integer  j  in 
the  bound 

|^(x)|  =  0(||x||-,)1  ||x||-»oo,  (57) 

is  as  large  as  possible,  subject  to  having  the  form  (51)  and  subject  to  the 
integral  condition  that  enough  freedom  remains  in  the  coefficients  {7*  |  ^G.Z£_i}  for 
it  to  be  possible  for  /wn  ip(x)dx  to  be  nonzero.  The  details  of  equations  (54)-(56) 
can  be  derived  from  the  Taylor  series  expansion  of  expression  (51)  for  large  ||x||  or 
from  the  relations  between  asymptotic  decay  rates  and  Fourier  transforms  that  are 
mentioned  in  Section  2.  Thus  one  finds  the  values  j  =  6,  j  =  7  and  j  =  7  in  the  bound 
(57)  for  the  n  =  2  thin  plate  spline,  the  n  =  3  linear  and  multiquadric  cases  and 
the  n  =  3  cubic  radial  basis  function  respectively.  Hence  the  right  hand  side  of  the 
diagonal  dominance  condition  (42)  is  finite.  Further,  when  q  and  a  have  the  values 
given  in  the  previous  paragraph,  it  seems  that  in  each  of  these  cases  the  0(-)  from 
the  linear  programming  calculation  actually  satisfies  inequality  (42),  provided  that 
c  is  not  too  large  in  the  multiquadric  radial  basis  function. 


The  integral  condition  that  has  just  been  mentioned  keeps  j  finite  in  inequality 
(57)  by  ruling  out  functions  0(-)  that  are  formed  by  applying  high  order  divided 
difference  operators  to  {^(||z||2)  |  i  6  Hn),  and,  more  importantly,  it  is  a  neces¬ 
sary  condition  for  inequality  (42)  to  hold  in  each  of  the  cases  (9).  To  justify  this 
statement  we  deduce  a  contradiction  if  we  have  the  diagonal  dominance  (42),  ip(-) 
being  an  absolutely  integrable  function  with  J-R"  tl>(x)  dx  =  0.  Inequality  (42)  allows 
the  Gauss-Seidel  method,  as  mentioned  in  the  first  paragraph  of  this  section,  to 
construct  a  function 

c[co\x)  =  £  4°°] 0(*  -  /),  xen\  (58} 

tezn 

that  satisfies  (C^00^)  =  6qi  |  feZ"},  and  the  sum  Y.ttzn  1*4°°' I  °f  moduli  of  coeffi¬ 
cients  is  finite,  which  implies  fnn  Cl°°)(x)dx  —  0.  The  cardinal  functions  C(  )  that 
are  studied  in  Section  2  for  each  of  the  cases  (9),  however,  have  the  property  that 
the  interpolation  formula  (5)  gives  s  =  /  when  /  =  1,  which  implies  fK „  C(x)  dx  =  1. 
Therefore  C^(-)  is  different  from  C(-).  Due  to  the  asymptotic  decay  properties  of 
C(-)  and  Cl°°l(*),  given  in  Section  2  and  deducible  from  the  smoothness  and  abso¬ 
lute  integrability  of  r/>(-),  we  now  have  a  contradiction  to  the  uniqueness  result  in 
Theorem  17  of  Buhmann  (1988b). 

It  follows  from  this  argument  that  inequality  (42)  is  satisfied  only  if  r/>(-)  is  not 
absolutely  integrable  or  if  fn„  4>{z)  dx  jk  0.  Remembering  that  the  number  of  terms 
of  Z  in  equation  (41)  is  finite,  and  that  <}>{■)  is  one  of  the  functions  (9),  the  first 
alternative  would  imply  that  the  right  hand  side  of  expression  (42)  is  a  divergent 
sum.  Therefore  diagonal  dominance  occurs  only  if  t/>(-)  is  absolutely  integrable 
with  a  nonzero  integral.  It  is  proved  in  Jackson  (1988),  however,  that  no  such 
function  exists  in  the  linear  case  <j>{r)  =  r  when  n  is  even.  It  also  follows  from 
his  analysis  that  condition  (42)  cannot  hold  for  the  linear,  cubic,  multiquadric  and 
inverse  multiquadric  functions  when  n  is  even  nor  for  the  thin  plate  spline  when  n 
is  odd.  Fortunately  this  lack  of  diagonal  dominance  does  not  prevent  satisfactory 
convergence  of  the  Gauss-Seidel  iteration  in  all  the  calculations  of  Section  4. 

For  example,  in  the  inverse  multiquadric  case  with  c  =  |  and  n  =  2,  the  only 
constraint  in  the  linear  programming  calculation  is  the  normalization  condition  (49). 
With  q  =  3  and  a  =  6,  the  final  value  of  the  objective  function  (50)  is  1.166,  which 
suggests  that  inequality  (42)  might  hold,  but  the  right  hand  side  of  this  expression  is 
infinite  because  YLizzl  It  —  0.0148.  One  can  deduce  from  the  bound  (47),  however, 
that  the  Gauss-Seidel  iteration  converges  for  values  of  b  up  to  15,  because  the 
calculated  rp(-)  satisfies  the  condition 

£  W*-m)|  <0.919,  meZ^,  (59) 

the  inequality  being  an  equation  when  m  =  0.  In  fact  the  largest  value  of  b  in 
these  calculations  is  b  =  24,  and  then  1^(01  =  1-810.  Even  in  this  case 
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Table  1  Some  values  of  Cf,(x)  when  <f>(r)  =  r  and  n  =  3 


X 

II 

*0 

6=5 

6  =  6 

h- 

II 

-O 

6=8 

(0.5, 0,0) 

0.399676 

0.399676 

0.399676 

0.399676 

0.399676 

(1.5, 0,0) 

-0.021868 

-0.021868 

-0.021868 

-0.021868 

-0.021868 

(2.5, 0,0) 

8.7094.4 

8.7095.4 

8.7095.4 

8.7095.4 

8.7095.4 

(3.5, 0,0) 

2.7883_6 

1.6869.6 

1.6721.6 

1.6733.6 

1.6733.6 

(4.5, 0,0) 

-2.4931.5 

-4.8588_« 

-5.1298.6 

-5.1386.6 

-5.1381.6 

(5.5, 0,0) 

-2.6420.5 

-6.6890.6 

7.3171.7 

6.9654.7 

6.9279.7 

(6.5, 0,0) 

-6.8362_6 

-8.6200.6 

-1.3422.6 

-5.7800.8 

-5.9653.8 

(7.5, 0,0) 

-5.0118.7 

-1.0026.6 

-1.5452.6 

-3.0887.7 

-3.3524_10 

(8.5, 0,0) 

1.6935.7 

-2.9567_8 

-2.3408.7 

-3.6663.7 

-7.4295.8 

the  convergence  of  the  Gauss-Seidel  iteration  is  entirely  satisfactory,  the  number  of 
iterations  to  reduce  the  maximum  residual  by  a  factor  of  10  on  the  49x49 grid  being 
about  300,  where  each  iteration  adjusts  at  most  8  of  the  coefficients  because 

the  set  V{j)  contains  at  most  8  elements.  If  necessary  we  could  have  increased  the 
values  of  q  and  a  to  ensure  the  convergence  of  our  iterative  procedure,  but  in  all 
cases  the  original  values  of  q  and  a  give  an  adequate  quasi-interpolating  function 

V’(-)- 

The  dependence  of  C*()  on  6  is  illustrated  in  Table  1  for  the  linear  radial  basis 
function  in  three  dimensions,  where  p,  denotes  px  10’.  In  this  case  C(-)  has  an 
exponential  asymptotic  rate  of  decay  of  the  form  (27),  while  satisfies  condition 
(57)  with  j  =  7.  Looking  across  the  rows  of  the  table,  we  see  that  C(,(x)  settles  down 
very  quickly  as  6  increases.  Therefore  much  of  the  last  column  reflects  the  decay  of 
the  true  cardinal  function  C(-).  The  later  entries  of  the  6=4  column,  however,  show 
the  algebraic  rate  of  decay  of  Cj,(-)  that  is  inherited  from  ip(-). 

To  conclude  this  section  we  address  the  goodness  of  the  approximation  Cj,(-)?» 
C(-),  keeping  in  mind  the  results  of  Table  1.  We  argue  that  the  decay  rate  of  the 
true  cardinal  function  provides  excellent  accuracy  in  Ct(x) «C(i)  for  0<  ||x||oo  <  |6, 
although  Ck(-)  may  decay  much  more  slowly  than  C(  ).  This  argument  depends  on 
a  conjecture,  which  in  it  strongest  form  asserts  that,  in  each  of  the  cases  (9),  every 
bounded  function  has  a  unique  bounded  interpolant  on  Zn,  namely  the  function  (5), 
where  C(-)  is  the  inverse  Fourier  transform  of  expression  (16).  An  equivalent  claim 
is  that  the  zero  function  is  the  only  bounded  function  in  our  space  of  approximations 
that  vanishes  on  Zn. 

Our  argument  requires  the  choice  of  x^(-)  and  the  convergence  of  the  Gauss- 
Seidel  iteration  to  be  such  that  (Cj(x)  |  x  G  7£n}  is  a  bounded  function,  and  we  also 
require  the  test  for  termination  of  the  iterative  procedure  to  be  so  fine  that  we  can 
suppose  that  Cj(-)  satisfies  the  equations 


ci(()  =  sot,  eez?. 


(60) 


I 


Then  the  error  function 


r6(x)  =  C(x)  -  C6(i),  x€Hn,  (61) 

is  uniformly  bounded,  and,  because  it  is  in  the  space  of  approximations,  the  conjec¬ 
ture  of  the  previous  paragraph  allows  rt(-)  to  be  equated  to  its  interpolant,  which 
is  the  identity 

rfc(*)=£r b(e)c(x-e),  ier,  (62) 

tezn 

the  right  hand  side  being  absolutely  convergent  because  of  the  decay  properties  of 
C(-)  that  are  given  in  Section  2.  Equations  (60)-(62)  and  {C(£)  =  Sot  \  t  £  Z£) 
imply  the  bound 

|r6(x)|  <  ||r6(-)||00  £  |C(x-£)|,  xe7i\  (63) 

and  we  draw  our  conclusions  from  the  fact  that  i  £  Z£  on  the  right  hand  side. 
The  main  points  are  that  only  small  values  of  |C(x  —  £)|  occur  in  this  absolutely 
convergent  sum  when  x  is  well  inside  the  box  ||z||°°  <6,  and  the  magnitudes  of  these 
small  terms  depend  not  on  the  decay  rate  of  Cj(-)  but  on  the  decay  rate  of  C(). 
Hence  the  true  cardinal  function  dominates  the  approximation  Cfc(x)  =  C(x)  — Tt(x) 
to  C(x)  when  0<  UxUoo  <  56,  but  Tt(x)  becomes  important  for  larger  values  of  HxHoo. 
For  example,  I\(-)  causes  the  final  increases  in  |Cj>(x)|  in  the  b— 7  column  of  Table 
1.  Moreover,  because  the  top  half  of  the  6=8  column  gives  excellent  approximations 
to  C(-),  we  see  that  the  contributions  from  F*,( - )  to  the  first  row  of  Table  1  are 
scaled  by  factors  of  at  most  10“5,  which  provides  the  consistency  across  this  row  of 
the  table. 

In  the  calculations  of  the  next  section  the  tolerance  that  stops  the  Gauss-Seidel 
iteration  is  set  to  such  a  small  number,  typically  10-16,  that  the  discrepancies  in 
equation  (60)  due  to  Cj,(-)^C^(-)  are  below  the  rounding  errors  of  the  given  tables. 
Further,  when  estimating  each  value  of  C(x)  we  increased  6  until  the  changes  to  Cj,(x) 
were  less  than  the  displayed  accuracy.  We  believe,  therefore,  that  the  tabulations  of 
C(-)  in  Section  4  are  close  to  the  true  cardinal  functions  of  interpolation,  and  that 
there  is  no  need  to  refer  back  to  the  details  of  the  approximations  of  the  algorithm 
of  this  section. 


4.  Numerical  results  and  discussion 

Table  2  presents  some  values  of  cardinal  functions  when  n  =  2  in  the  linear,  mul¬ 
tiquadric,  inverse  multiquadric  and  thin  plate  spline  cases.  The  lower  precision  in 
the  penultimate  column  is  due  to  the  rather  slow  convergence  of  the  sequence  of 
approximations  {Ct(  )  |  6  =  1,2,3, .. .}  when  <f>(r)  =  (r2  +  ^)-,/2.  From  a  practical 
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Table  2  Some  values  of  C(x)  when  n  =  2 


X 

II 

*1 

(r2  +  i),/J 

(r2  +  I)-V2 

r2  log  r 

(0.5,0) 

0.434190 

0.560007 

0.473146 

0.535938 

(1.5,0) 

-0.018506 

-0.090066 

-0.035897 

-0.074666 

(2.5,0) 

-1.0044_3 

0.018726 

3.4454-3 

0.013094 

(3.5,0) 

-4.6723_5 

-4.0942-3 

-1.7455-4 

-2.4549-3 

(4.5,0) 

-2.7134.5 

9.2458.4 

8.525-5 

4.8375-4 

(5.5,0) 

-1.0850-s 

-2.1614-4 

3.095-s 

-9.8539-s 

(6.5,0) 

-4.7241-6 

5.0800-s 

2.114-s 

2.0522-s 

(7.5,0) 

-2.3377-6 

-1.2491-s 

1.382-s 

-4.3399-6 

(8.5,0) 

-1.2608-e 

2.9014-6 

9.64-e 

9.2796-7 

(9.5,0) 

-7.2707-7 

-7.9513.7 

6.97-e 

-2.0006-7 

(10.5,0) 

-4.4266-7 

1.4590-7 

5.20_6 

4.3408-8 

point  of  view  the  most  important  question  is  often  ‘what  is  the  greatest  value  of 
||x||  such  that  |C(x)|  >  10-6  because  for  all  larger  values  of  ||x||  one  may  be  able 
to  treat  C{x)  as  negligible.  Table  2  shows  that,  according  to  this  criterion,  the  thin 
plate  spline  is  marginally  better  than  the  linear  and  multiquadric  radial  basis  func¬ 
tions,  while  the  inverse  multiquadric  is  least  good.  If  larger  values  of  |C(x)|  can  be 
neglected,  then  <£(r)  —  r  seems  to  be  the  best  of  the  four  given  cases,  but  c  =  |  is  the 
only  c  of  this  tabulation  of  multiquadric  and  inverse  multiquadric  cardinal  functions. 
If  one  preferred  <f)(r)  =  (r2  +  y^)1/2,  for  example,  then  one  would  have  localization 
properties  that  tire  similar  to  those  of  4>{r)  =  r  and  one  would  have  differentiability 
too. 

Because  every  x  in  Table  2  is  on  the  first  coordinate  axis,  Figures  1-4  present 
some  properties  of  the  cardinal  functions  on  7Z2.  Each  figure  is  constructed  from 
the  values  of  C()  on  the  201  x  201  square  grid  with  mesh  size  0.1  on  the  square 
IMloo  <  10.  For  each  of  the  heights  {h  =  10“*  |  k  =  1, 2, 3, . . .},  we  calculated  the 
convex  hull  of  the  points  of  this  mesh  at  which  |C(x)|  >  h.  The  piecewise  linear 
curves  in  the  figures  are  the  boundaries  of  these  convex  hulls  in  the  fourth  quadrant, 
the  range  of  x  being  HxH^,  <  8.  The  shaded  regions  of  the  figures  are  the  spaces 
between  the  h  =  10-5  and  h  =  10“6  curves,  which  show  clearly  that  the  inverse 
multiquadric  radial  basis  function  is  the  least  successful  of  the  four  cases. 

For  each  radial  basis  function  <£(•),  the  spacings  between  the  convex  hull  bound¬ 
aries  of  the  heights  {/i  =  10~*  |  k  =  1,2, 3, ...}  indicate  the  asymptotic  decay  rate 
of  the  cardinal  function  of  interpolation,  but,  particularly  in  Figure  2,  the  range 
Halloo  5; 8  is  rather  small  for  this  purpose.  When  the  cardinal  function  decays  expo¬ 
nentially,  which  implies  a  bound  of  the  form  (27),  then  the  average  spacing  tends 
to  a  constant,  and  this  feature  is  shown  well  in  Figure  4.  In  the  other  three  cases, 
however,  the  asymptotic  decay  rate  is  algebraic  as  suggested  by  inequality  (25),  so 
the  spacings  between  convex  hull  boundaries  tend  to  diverge.  This  property  is  clear 
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L  Table  3  Further  values  of  C(x)  when  n  =  2 


X 

<t>(r)  =  (r2 

+  i)1/2 

<*>(r) 

=  r 

C(x) 

INI  \C(x) 

C(x) 

llxlli  C'(x) 

(10.5,0) 

1.4590.7 

0.01862 

-4.4266.7 

-0.05650 

(11.5,0) 

-6. 8753_g 

-0.01382 

-2.8178.7 

-0.05668 

(12.5,0) 

-4.396.9 

-0.00134 

-1.8618.7 

-0.05682 

(13.5,0) 

-1.365.8 

-0.00612 

-1.2698.7 

-0.05694 

(14.5,0) 

-6.928.9 

-0.00444 

-8.899.8 

-0.05704 

(15.5,0) 

-5.698.9 

-0.00510 

-6.386.8 

-0.05713 

in  Figures  1  and  3,  but  the  multiquadric  figure  is  more  interesting  because  of  its 
similarity  to  the  exponential  decay  of  the  thin  plate  spline  for  h  >  10~6. 

The  shapes  of  the  convex  hull  boundaries  also  deserve  comment.  We  know  from 
equation  (31)  that  they  tend  to  be  circular  in  the  linear  and  multiquadric  cases, 
and  we  see  this  feature  in  Figure  1.  Asymptotic  circularity  also  occurs  for  inverse 
multiquadrics  (Buhmann,  1988b),  but  the  range  HxHoo  <  8  is  too  small  for  it  to 
appear  in  Figures  2  and  3.  In  Figure  4,  however,  where  the  asymptotic  decay  rate 
of  C(-)  is  exponential,  it  seems  that  the  convex  hull  boundaries  tend  to  be  diamond 
shaped  (||x||i  =  constant).  This  property  is  obtained  in  cubic  spline  interpolation 
on  2",  because  the  exponential  rate  of  decay  along  any  line  that  is  parallel  to  a 
coordinate  direction  is  independent  of  the  displacement  of  the  line  from  the  origin. 
Perhaps  a  similar  property  holds  for  radial  basis  function  interpolation  methods  with 
exponential  decay,  in  which  case  it  would  be  most  appropriate  to  let  ||x||  =  ||x||i  in 
the  bound  (27). 

The  figures  so  far  fail  to  reflect  the  asymptotic  behaviour  of  C(-)  in  the  multi¬ 
quadric  case  <t>(r)  =  (r 2  +  5)1/2.  Indeed  the  signs  of  the  entries  in  the  middle  column 
of  Table  2  alternate,  but  the  theory  of  Section  2,  in  particular  expressions  (31)  and 
(38),  show  that  C(x)  is  negative  whenever  ||x||  is  large  and  x  is  away  from  the 
points  of  the  grid  2n.  Therefore  Table  3  provides  some  more  values  of  this  cardinal 
function.  Now  we  see  that  the  dominant  part  of  C(x)  can  be  a  negative  multiple  of 
||x||j5,  and  that  there  is  fair  agreement  with  the  factor  -0.005212  in  equation  (39). 
It  is  easier,  however,  to  use  tabulated  values  of  C(-)  to  estimate  the  constants  of 
expression  (32)  in  the  <f>(r)  =  r  case,  partly  because  the  first  derivative  discontinu¬ 
ities  of  the  radial  basis  function  allow  C(x)  to  be  nonpositive  for  all  moderate  values 
of  ||x||.  Thus  the  last  column  of  Table  3  shows  good  agreement  with  the  number 
—0.05747.  In  order  to  support  the  other  constant  of  expression  (32),  we  note  that 
at  x  =  (9.5,9.5),  for  example,  we  have  C(x)  =  -1.54xl0-7  and  ||x||| C7(x)  =  -0.0672. 

Some  values  of  cardinal  functions  in  three  dimensions  are  given  in  Table  4.  The 
asymptotic  decay  rate  of  C(  )  when  4>(r)  =  r  is  particularly  impressive,  being  about 
a  factor  of  10  per  data  point.  We  recall  from  the  last  paragraph  of  Section  2  that 


Table  4  Some  values  of  C(x)  when  n  =3 


X 

<t>{r)  =  r 

v1  + 

(r!  +  i)1'1 

4>{r)  =  r 3 

(0.5,0, 0) 

0.399676 

0.439796 

0.545232 

0.560447 

(1.5, 0,0) 

-0.021868 

-0.034425 

-0.082773 

-0.099110 

(2.5, 0,0) 

8.7095.4 

3.2618.3 

0.017282 

0.025761 

(3.5, 0,0) 

1.6733.6 

-3.5220.4 

-3.9703.3 

7.5461.3 

(4.5, 0,0) 

-5.1381.6 

4.3009_s 

9.8001.4 

2.3896.3 

(5.5, 0,0) 

6.9300.7 

-5.7089.6 

-2.5541.4 

-7.9975.4 

(6.5,0, 0) 

-6.1395.8 

7.9934.7 

6.9290_s 

2.7841.4 

(7.5, 0,0) 

3.5619.9 

-1.1589.7 

-1.9363.5 

-9.9717.5 

(8.5, 0,0) 

-1.9969_n 

1.7210.8 

5.5319_6 

3.6474.5 

(9.5, 0,0) 

-2.8859.H 

-2.6008.9 

-1.6073.6 

-1.3556.5 

(10.5,0,0) 

4.8041.12 

3.9834_,o 

4.7315.7 

5.1017.6 

the  multiquadric  cardinal  function  decays  like  ||x||jn  as  ||x||  — >  oo,  and  we  see  in 
the  table  that  this  algebraic  rate  is  sufficiently  strong  for  the  localization  proper¬ 
ties  of  multiquadric  interpolation  with  moderate  c  to  compare  favourably  with  the 
exponential  decay  of  the  <f>{r)  =  r3  case.  Therefore,  if  one  cannot  tolerate  the  first 
derivative  discontinuities  in  the  interpoiant  when  <f>(r)  =  r,  it  may  be  better  to  turn 
to  multiquadric  radial  basis  functions  instead  of  (f>{r)—r3,  but  the  obvious  choice  for 
a  regular  grid  is  cubic  spline  interpolation.  In  large  calculations  the  amount  of  work 
of  cubic  spline  methods  is  orders  of  magnitude  less  than  that  of  radial  basis  function 
techniques,  but  perhaps  the  introduction  of  transputers  and  further  research  will 
alleviate  this  gap.  Therefore  we  note  that  the  cubic  spline  decay  rate  of  (2— \/3)  per 
data  point  is  less  good  than  the  localization  properties  when  <f>(r)  =  r,  and  that  the 
comparison  with  multiquadrics  depends  on  c.  All  of  these  methods  reproduce  cubic 
polynomials  in  the  theoretical  case  of  interpolation  on  an  infinite  grid,  but  only  the 
cubic  spline  has  this  property  when  the  grid  is  finite. 

The  relevance  of  approximation  on  an  infinite  grid  to  finite  grids  has  been  studied 
by  Dyn  and  Jackson  (private  communication,  1988).  It  is  important  not  only  be¬ 
cause  finiteness  is  necessary  in  practical  calculations  but  also  because  of  the  gains  in 
efficiency  that  may  be  achievable  by  partitioning  very  large  finite  grids  into  smaller 
ones.  Therefore  we  give  it  some  attention  now.  As  at  the  end  of  Section  3,  we  relate 
an  interpoiant  s&(-)  on  the  finite  grid  Z£  =  Zn(~\[— b,  6]n  to  our  interpoiant  (5)  on  the 
infinite  grid  Zn.  The  equation 

st(x)  =  s(x)  -  7?(x),  x€ftn,  (64) 

defines  a  function  tj(-)  that  is  in  the  space  of  approximations.  Therefore,  assuming 
a  suitable  uniqueness  condition  for  interpolation  on  Zn,  we  may  equate  rj(-)  to  its 


interpolant,  which  yields  the  identity 


*(*)-*(*)=  £  {/(0  -  sk(i)}  c{x  - 1),  xenn.  (65) 

tez"\z » 

Even  if  /(•)  is  bounded,  the  magnitude  of  s/,(£)  for  large  ||^||  depends  on  the  growth 
of  <f>(r)  as  r  — ►  oo.  Thus,  corresponding  to  the  derivation  of  inequality  (63)  from 
equation  (62),  we  find  a  bound  of  the  form 

|s(x)-sfc(x)|</?  £  {1  +  ^(||£||2)}|C(x-£)|,  rer,  (66) 

where  (3  is  a  constant,  the  sum  being  absolutely  convergent  for  till  of  the  cardinal 
functions  that  are  considered  in  Section  2.  Again  all  the  factors  |C(x  —  £)|  are  small 
when  x  is  well  inside  the  box  ||x||oo  5;  b.  Hence  the  extent  to  which  S(,(x)  enjoys 
the  properties  of  s(x)  depends  on  the  distance  from  x  to  the  set  2n\Z„-  If  n  =  3 
and  <j>(r)  =  r,  for  example,  Table  4  suggests  the  strong  result  that  |s(x)  —  S{,(x)|  is 
O(10-1<),  where  d  =  min{||x  —  ^||oo  j  i  6  2n\Z™},  so  there  is  rapid  decay  in  the 
perturbations  to  the  approximation  from  the  finiteness  of  the  grid. 

Tables  2-4  give  excellent  support  to  the  assumptions  on  uniqueness  of  interpola¬ 
tion  that  are  crucial  to  equations  (62)  and  (65).  Here  we  have  in  mind  that,  for  each 
of  the  radial  basis  functions  (9),  the  inverse  Fourier  transform  of  the  function  (16) 
provides  a  unique  cardinal  function  of  the  form  (6)  whose  coefficients  {ct  \  Zn} 
satisfy  the  asymptotic  conditions  of  Theorem  17  of  Buhmann  (1988b).  These  con¬ 
ditions  correspond  to  the  decay  of  (C(x)j  as  ||x||  — ►  oo  that  is  studied  in  Section  2. 
If,  however,  the  equations  {C{£)  =  6at  I  £  G  Zn}  failed  to  define  a  unique  bounded 
C(  )  of  the  form  (6),  then  different  attempts  to  calculate  C(-)  might  yield  different 
cardinal  functions.  Now  the  method  of  the  algorithm  of  Section  3  has  nothing  to 
do  with  the  inversion  of  Fourier  transforms,  and  we  even  apply  the  Gauss- Seidel 
iteration  in  some  cases  when  the  diagonal  dominance  (42)  is  not  obtained.  Thus 
it  is  likely  that  our  algorithm  would  find  a  cardinal  function  that  is  different  from 
the  inverse  transform  of  C(  )  if  it  were  possible  to  do  so.  Tables  2-4,  however,  seem 
to  confirm  the  asymptotic  properties  that  are  established  in  Section  2.  Further, 
when  ip(-)  satisfies  the  diagonal  dominance  condition  (42),  the  proof  of  Lemma  10 
of  Buhmann  (1988a)  shows  that  the  Gauss-Seidel  iteration  of  Section  3  gives  the 
cardinal  function  that  is  the  inverse  Fourier  transform  of  C(  ).  Therefore  we  believe 
that  the  uniqueness  assumptions  are  true. 

Although  the  algorithm  of  Section  3  is  suitable  for  calculating  the  cardinal  func¬ 
tions  that  we  have  studied,  it  is  not  recommended  for  interpolation  to  a  general 
function  on  a  finite  grid.  Our  reasons  include  the  loss  of  equivalence  symmetry  and 
the  need  for  a  range  of  quasi-interpolating  functions  ip(-)  in  order  to  take  account 
of  the  effects  of  edges  of  the  grid.  Therefore  some  other  algorithms  are  currently 
under  investigation  at  Cambridge,  particularly  the  use  of  preconditioned  conjugate 


gradient  methods  for  solving  the  interpolation  equations.  Such  methods  have  al¬ 
ready  been  applied  successfully  by  Dyn,  Levin  and  Rippa  (1986)  when  the  data  are 
not  confined  to  a  regular  grid. 

In  conclusion  we  note  that  the  asymptotic  properties  of  cardinal  functions  are  so 
encouraging  that  many  more  questions  deserve  attention.  Here  are  three  examples. 
If  we  estimate  the  interpolant  s(i)  in  a  way  that  ignores  the  function  values  {f(?)  \ 
f  €  2n }  for  large  ||£  —  x||,  is  there  a  good  way  of  making  the  estimate  a  continuous 
function  of  x?  If  we  extend  the  space  of  approximating  functions  to  include  low  order 
polynomials  explicitly,  can  the  interpolant  on  2”  provide  not  only  reproduction  of 
these  low  order  polynomials  but  also  the  localization  properties  that  are  noted  in  the 
paragraph  that  includes  expressions  (64)-(66)?  Although  the  analysis  of  Section  2  is 
an  application  of  Fourier  techniques,  do  the  main  conclusions  of  this  section  require 
interpolation  on  a  regular  grid?  Clearly  there  is  much  practical  and  theoretical 
work  to  be  done,  and  it  seems  probable  that  it  will  yield  some  very  powerful  new 
algorithms  for  multivariable  approximation. 
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Abstract  The  Fourier  operator  of  even  order  S  is  defined  as  a 
-  n 

projection  of  the  family  of  continuous  2TT-antiperiodlc  functions 
onto  the  trigonometric  polynomials  of  half-angles.  It  is  shown  that 
this  operator  is  a  natural  extension  of  the  classical  Fourier  opera¬ 
tor  and  corresponds  to  the  case  where  the  dimension  of  the  projec¬ 
tion  subspace  is  even.  The  operator  is  used  to  extend  a  result 
of  Szabados,  concerning  optimal  choice  of  nodes  for  trigonometric 
interpolation,  to  the  case  of  an  even  number  of  nodes  as  well  as  to 
the  case  of  complex  interpolation  by  algebraic  polynomials  on  the 
unit  circle . 

Key  words:  Fourier  operator.  Minimum  norm  projection,  Trigonometric 
interpolation. 
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1.  The  Fourier  operator  of  even  order 


Various  problems  of  approximation  theory  involve  the  trigonometric 
polynomials  of  half- angles 

n  fl 

h  (e)  *  Z  rc  cos(2k-l)T  +  d.  sin(2k-l)— ] 

n  k=l  ^  £  K  Z 

Properties  of  these  polynomials  and  their  application  to  interpo¬ 
lation  and  to  the  theory  of  quadrature  were  considered  by  Tureckii 
(1960,1968).  Recently  (see  Brutman  and  Pinkus  (1980))  the  polynomials! 

h  (0)  were  found  to  be  useful  in  proving  the  Erdos  conjecture  concern- 
n 

ing  minimal  norm  interpolation  on  the  unit  circle  for  even  number  of 
points.  In  the  following  the  trigonometric  polynomials  of  half¬ 
angles  are  used  in  order  to  define  an  even-order  analogue  of  the  clas¬ 
sical  Fourier  operator. 

Let  be  the  Banach  space  of  functions  continuous  and  antiperiodic 

on  [0,2ir)  (i.e.  satisfying  f(0)  *  -f(2ir)),  equipped  with  the  uniform 

norm,  and  denote  by  H  the  following  2n-dimensional  subspace  of  C„ 

n  2  it 

H  *  span(sin|-,  cos|-  ,...,sin^2^  cos--2°~^6} 

n  2  2  2  2 


It  is  known  that  is  an  orthogonal  system  on  [0,2tt)  . 
type  operator  :  C  is  defined  by 

H  Q  Q 

(Snf)(0)  -  Z  {ckcos(2k-l)j  +  dksin(2k-l)|-}. 


The  Fourier- 


;  where 


1  f  ft 

\  -  ff  f(6)cos(2k-l)|  d0, 

1  2lr  0 

dk  -  i  /  f (6) sin(2k-l)y  d0. 


The  following  integral  representation  of  S  holds 

n 


Theorem  1 


(Sf) (0)  -—  /  f (t)D  (T-0)dt , 
n  w0 


where 


~  5  sin  n6  ... 

Dn(6)  "  kf1COs(2k-1)l  “  2sin(9/2)  *  (4) 

Note  that  the  kernel  D  (9)  in  (3)  is  a  natural  extension  of  the 

n 

classical  Dirichlet  kernel  D  (6)  **  sin(m9/2)/2sin(9/2) ,  m*  2n+l, 

m 

and  corresponds  to  the  case  of  even  m.  As  a  direct  consequence  of  the 
representation  (3)  we  get 
Corollary  1 


II S  II-  -  /,/2-l’*-°;nei-d9  5  i  .  (5) 

11  n"  ir  J  sin9  n 

■  Remark  To  the  best  of  our  knowledge,  the  quantities  p  appear  for 

j  .  n 

the  first  time  in  Szego  (1921):  "Mann  kann  die  Konstanten 


r  / 

m  w  QJ 


[sin  m8| 
sin 


d9  ,  m  ■  0, 1,2, . . , 


betrachten,  welche  sich  fur  ungerade  m  auf  die  Lebesgueschen  Kon¬ 
stanten  reduzieren . "  Szego  has  proved  the  following  representation 

rm  T  f/T  3  +  5+  **•  +  2^]4^T  »  (6) 

v*l 


j which  was  used  by  Galkin  (1971)  to  obtain  the  precise  estimate 


0.9894 


4 

<  r  -  ~r  log  m  <  1 . 


jThe  quantities  Pq  were  also  used  by  Geddes  and  Mason  (1975)  as  an 
{upper  bound  for  the  norm  of  the  Taylor  projection  of  even  order. 

!  They  gave  the  following  formula 


(8) 


4  “  _ 1  „  2k-l 

0  -  —  1  tt: — r  tan  -  tt 

n  tt  .  .  2k-l  4n 

k»l 

which  is  analogous  to  the  representation  of  the  classical  Lebesgue 
constants  due  to  Fejer  (1910). 

Continuing  the  analogy  with  the  classical  case  we  define  the  Fejer- 

type  operator  F  :  C„  -+■  H  to  be  the  arithmetic  mean: 

1  r  r  n  2tt  n 


F  *  ~{S.  +  S0  +  ...  +  S  } 
n  n  1  2  n 


Then  we  have 
Theorem  2 


(F.  0(6)  m  TZ  f  f(T)i(T-9)dT, 
n  nr  '  n 


where 


.  n+1  .  .  n  0 

sin  ~y~  6  sin  — 
_____ 

2sin  j 


Note  that  in  contrast  to  the  classical  case,  the  kernel  a  (6)  is 

n 

not  positive.  Yet  this  is  not  surprising,  since  the  operator  Fn  is 

defined  on  the  space  which  does  not  contain  positive  functions. 

This  observation  raises  the  natural  question  of  whether  every  feC^ 

:  can  be  uniformly  approximated  by  the  Fejer-type  polynomials  F^(9)  . 

It  can  be  shown  by  applying  the  classical  result  of  Fejer  (1904) 

|  that  the  answer  to  this  question  is  affirmative,  namely 

Theorem  3  The  set  H  *  (cos(0/2) , sin(9/2) , cos(30/2) , sin(30/2) , . . . } 

is  dense  in  C,  . 

2tt 

Applying  Theorem  3  and  arguments  analogous  to  those  used  for  the 

classical  trigonometric  polynomials,  the  following  analog  of  the 

Marc inkiewicz-Zygmund -Berman  identity  can  be  proven. 

i  Theorem  4  Let  P  be  any  projection  from  C„  onto  H  .  Then 
.  - - —  n  ztt  n 


~  /  P  (f(x-T),0+T)dT  -  (S  f)(6) 


1 


rj  ■£*  ui  C7>  -J  CD  kO 


As  a  consequence  of  the  above  theorem  we  have  the  following  minimal 
norm  property  of  (For  the  corresponding  classical  results  see 
Losinskii  (1948)  and  Cheney  et  al.  (1969)). 

Corollary  2 


ISJI  _<  HPnl 


Moreover,  the  equality  in  (13)  holds  only  if  P  =  S 

n  n 


2.  An  interpolation-theoretical  extremum  problem. 


Tureckii  (1968)  posed  the  following  problem:  Let  0<9  <9  < . .  .<0„  <2ir 

0  1  2n 

and  let  t.O)*^  n(0)»  k“0, 2n  be  the  fundamental  trigonometric 

polynomials  of  degree  n,  i.e.  such  that  t  (0.)=6  ,  k,j=0,l, _ ,2n. 

2n  k  J 

For  what  system  of  nodes  {9,  },  „  will 

k  k=0 


S  ■  VVV 


2  ?r  2n 

,0  )  »  /  {  E  k(0)|P}d0,  (0<p<~)  (14) 

0  k-0 


be  minimal?  It  was  conjectured  by  Tureckii  that  the  optimal  solution 
corresponds  to  the  equidistant  nodes.  Schumacher  (1976)  proved  this 
conjecture  in  case  p=l,  as  a  corollary  of  a  much  more  general  theo¬ 
rem.  Szabados  (1980)  found  a  direct  solution  to  the  problem  in  the 
case  13><“>,  specifically  he  proved  the  following 

Theorem  5  Let  p  >_1.  The  integral  I  (9., 9. , . .  ., 9.  )  is  minimal  iff 

p  u  l  zn 

the  nodes  are  identical  with  the  equidistant  nodes  9k*2kir/(2n+l) , 
k*0, 1, . .  .,2n,  or  with  their  translation.  Moreover, 


min  I  (9  ,9  ,  ...,9  )  - — ■ y  / 1 D  (9) |  Pd9 

9k  P  °  1  2n  (2n+l)P_1  0  n 


i where  D  (9)  is  the  classical  Dirichlet  kernel. 

I  n 

In  the  following  we  extend  the  above  result  to  the  case  where  the 

inumber  of  interpolation  nodes  is  even.  Leu  0 <_9  <9.< . . . <0„  <  2tt 
_  1  2  2n 

land  denote  by  n(0)*  k»l,2, . . . ,2n,  the  fundamental  trigono- 

|  metric  polynomials  of  half-angles,  i.e.  t.  t  H  and  t.(0  )"6  . .  Let 

k  n  k  j  icj 


2  rr  2r» 

i  -  i  (e  ,e  )  *  /  {  e  1 1.  (e) |p}de  (16) 

p  P  1  ^  zn  0  k«l  k 

The  following  result  holds 

Theorem  6  The  Integral  I  (9. ,6„,  .. .,6„  ),  p  >  1,  is  minimal  iff 
—————  p  l  l  zn  - 

the  nodes  are  identical  with  9^  ■  kir/n,  k*0, 1, .  . .  ,2n-l,  or  with 
their  translation.  Moreover 

_p  2w 

min  I  (9  ,0  ,...,0  )  =  — ^ - -  /  ]d  ( 8)  j Pd9,  (17) 

\  P  1  2  2"  (2n)p_2  0  " 

where  D^(9)  is  the  Dirichlet  kernel  corresonding  to  the  trigonometric 
polynomials  of  half-angles.  The  proof  of  this  theorem,  which  is 
based  on  (12),  follows  the  same  reasoning  as  the  proof  of  Szabados. 
Our  final  result  concerns  complex  interpolation  by  algebraic 

polynomials  on  the  unit  circle.  Let  z^exp ( i8 ^) ,  k=0,l . m  be 

nrt-1  distinct  points  and  let  1,  (z),  k»0, . .  be  the  corresponding 

k  y  in 

fundamental  polynomials.  It  was  proved  by  Brutman  (1980)  that 


\,m(e  W‘2n 


|tk,n<e> ! »  nr2n-l 


Combining  (18)  with  Theorems  4  and  5,  we  arrive  at  the  following 
Theorem  7  The  integral 


VVh . *  l  (k20|lk,»“i6>l'’M9lI>  -  1  (1,) 

is  minimal  iff  z,  -  exp  [ikir/(m+l)+a  3,  k-0,l,...,m,  some  a.  Itoreover 


2P  2it 

min  Ip (zQ, zm)  *  '~p-i  [  ^m<e>  1  de 


(nrt-ir  0 


jwhere  D  (9)*sin(m9/2) /2sin(0/2)  is  a  generalized  Dirichlet  kernel. 

i  m 
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ON  MULTIVARIATE  POLYNOMIAL  INTERPOLATION 


N.  DYNt,  A.  RON* 

* School  of  Mathematical  Sciences,  Tel  Aviv  University 
‘Department  of  Computer  Science,  University  of  Wisconsin-Madison 


Abstract  A  class  of  spaces  of  multivariate  polynomials,  closed  under  differentiation,  is 
studied  and  corresponding  classes  of  well  posed  Hermite-type  interpolation  problems  are 
presented.  All  Hermite-type  problems  are  limits  of  well  posed  Lagrange  problems. 

The  results  are  based  on  a  duality  between  certain  spaces  of  multivariate  exponential- 
polynomials  H  and  corresponding  spaces  of  multivariate  polynomials  V,  used  by  Dyn  and 
Ron  ( 1988)  to  establish  the  approximation  order  of  the  span  of  translates  of  exponential 
box  splines.  In  the  interpolation  theory  V  is  the  space  of  interpolating  polynomials  and 
H  characterizes  the  interpolation  points  and  the  interpolation  conditions,  both  spaces 
being  defined  in  terms  of  a  set  of  hyperplanes  in  JR* . 

This  geometric  approach  extends  the  work  of  Chung  and  Yao  (1977)  on  Lagrange 
interpolation,  and  also  a  subset  of  the  Hermite-type  problems  considered  via  the  Newton 
scheme,  by  several  authors  (see  Gasca  and  Maetzu  (1982)  and  references  therein).  For  a 
different  approach  to  the  interpolation  problem  see  Chui  and  Lai  (1988). 

It  is  the  systematic  and  unified  analysis  of  a  wide  class  of  interpolation  problems 
which  is  the  main  contribution  of  this  paper  to  the  study  of  multivariate  polynomial 
interpolation. 

Keywords:  Multivariate  interpolation,  multivariate  polynomials,  Hermite-type  interpo¬ 
lation. 


1.  The  Interpolating  Polynomial  Spaces 


The  spaces  of  interpolating  polynomials  we  consider  here  are  more  general  than  the 
total  degree  polynomials  xm  (polynomials  of  degree  <  m),  and  are  still  closed  under 
differentiation. 

Given  a  set  of  directions  A  =  {al,...,an}  C  JR*,  with  the  property  spanA  =  JR*, 
consider  the  space  of  polynomials 


V(A)  =  span 


(r 


x)  ieS(A)\ 


where  5(A)  consists  of  index  sets  corresponding  to  “small  enough”  subsets  of  A,  namely 
5(A)  ={/c{l,...,n}|  span{a*  |  i  *  1}  =  JR*}  .  (2) 

By  choosing  I  €  5(A)  such  that  {1, . . . ,  n}\J  is  a  basis  of  2R*,  we  conclude  that 

'P(A)  C  x n— *  .  (3) 

To  see  that  V{A)  is  closed  under  differentiation,  observe  that 

A  JI(a<  •  x)  =  Y.  ai  II(a<  ’  x>  ’  (4) 

°  1  «€/  *€/ 

and  that  if  I  €  5(A)  then  any  subset  of  /  is  in  5(A). 

A  more  involved  analysis  is  required  in  order  to  show  the  following  two  properties 
of  V(A),  demonstrated  in  Dyn  and  Ron  (1988): 

(a)  Let  d  =  d(A)  =  min{|/|  |  /  C  {1,. . .  ,n}  ,  I  $  5(A)}.  Then  C  P(A). 

(b)  The  dimension  of  V{A)  equals  the  number  of  bases  that  can  be  formed  from  A. 
Combining  (a)  and  (3)  we  conclude  that 

xd-x  C  V{A)  C  .  (5) 

If  a1,..., a"  are  in  “general  position”,  namely  any  s  vectors  among  a1,..., a"  form  a 
basis  of  1R*,  then  it  is  easy  to  see  that  d  -  n  -  s  +  1.  Hence  V(A)  = 

To  introduce  a  basis  of  V(A),  consider  n  hyperplanes 

Hi  =  {x  €  IR*  |  a*  •  x  =  7<}  ,  t  =  l,...,n,  (6) 


determined  by  T  =  (71, . . .  ,7n)  €  JRn ■  For  each  v  £  JR*  define 

U  =  {»  €  {l,...,n>  |  v  €  Hi)  . 


(7) 


and  consider  the  set  of  intersection  points  of  Hn , 


C(A,r)  =  {v€l'|  span  Av  =  JR*}  ,  Ar  =  {&' \  i  £  Iv}  .  (8) 

Choosing  T  so  that  |/v!  =  s  for  v  £  V(  A.  T),  where  j/v|  denotes  the  cardinality  of  Jv,  we 
conclude  from  ( b)  that 

dimV(A)  =  |I/(A,r)|  .  (9) 

Furthermore,  the  following  polynomials 

*<*>  -  n  ■  vev<A- r>’  no) 

are  linearly  independent,  since 

f  0  u  ^  v  , 

/»v(u)={  V.u  6  V(.4.D  ,  (11) 

l  1  U  =  V  . 

and  hence  constitute  a  basis  of  V(A). 

The  pair  (4,T)  is  termed  “simple”  (for  simple  intersection  points  as  opposed  to 
multiple  ones)  if  |/v|  =  s  for  all  \  £  V(AS). 


Remark  1  It  is  shown  by  Ron  (1988)  that  for  fixed  A  the  set  of  all  T  £  IRn  such  that 
(A,T)  is  simple,  is  dense  in  IRn. 

The  explicit  form  (10)  of  a  basis  of  V(A)  indicates  that  the  following  result  holds. 


Proposition  1  V(A)  consists  of  polynomials  of  degree  <  n  —  s,  which  are  of  degree 

<  n  -  a  -  |{i  €  {1 . «}  I  a'  €  span{y}}|  +  1  along  hyperplanes  of  the  form  y  •  x  =  A, 

y  €  A,  A  €  1R.  Furthermore,  let  Y  =  {y1....^*},  be  k  <  s  pairwise  distinct  directions 
in  A.  Then  the  degree  of  any  p  £  V{A)  along  the  intersection  of  k  hyperplanes  of  the 
form 

y3-x=pj  .  £  R  ,  (12) 

is  at  most 

n-s  — |{i6{l . n}  |  a’  6  (Y)  }|  +  dim  (Y)  .  (13) 


where  (Y)  =  spanT. 


Proof  Since  for  (A,  T)  simple,  and  v  €  V(A,  T),  {a*  |  i  €  /»}  is  a  basis  of  2R*,  each  p,  in 
(10)  consists  of  at  least  |{i  €  {l,...,n}  |  a*  €  span{y}}|  -  1  factors  which  are  constant 
along  y  •  x  =  A,  A  €  JR.  Similarly,  one  can  count  the  constant  factors  in  p,  of  (10)  along 
the  intersection  of  the  hyperplanes  (12),  to  conclude  (13). 


Remark  2  The  space  A)  consists  of  all  polynomials  over  IR*  with  the  properties  stated 

in  Proposition  1.  This  will  be  shown  elsewhere. 


2.  The  Interpolation  Problems 


In  this  section  we  present  a  class  of  interpolation  problems  which  are  uni  solvent  in  V(A) 
for  fixed  A.  The  interpolation  points  and  the  data  at  each  point,  which  is  of  Hermite 
type,  are  determined  by  the  choice  of  T  =  (7i,...,7n)  €  2Rn.  The  set  of  interpolation 
points  consists  of  all  points  of  intersection  of  at  least  s  of  the  hyperplanes  (6),  namely,  it 
is  the  set  denoted  by  V(A,T).  To  define  the  interpolation  conditions  at  each  v  6  V(A,  T), 
we  consider  the  set  of  directions  related  to  v 

At  =  {•*  |  *  €  /*}  ,  (14) 

and  a  corresponding  polynomial  space  defined  by 

£(At)  =  {p  €  *  |  [ !!(«'  ■  D))p  =  0  ,  /  i  5(At)}  ,  (15) 

•6/ 

where  D  =  . . . ,  jfj)-  Since  each  I  in  (15)  satisfies  |/|  >  dT  =  d(AT),  it  is  dear  that 

r^,_j  C  /C(AV).  The  space  /C(AT)  is  closed  under  differentiation  since  Dm  commutes 
with  any  polynomial  in  D.  In  terms  of  K(Ar)  the  interpolation  conditions  at  v  are 

fa(£)p](v)  =  [?(Z?)/](v)  ,  q  €  K(AV)  ,  (16) 

where  /  is  smooth  enough,  and  q(D)  is  obtained  from  the  polynomial  q(x)  by  repladng 
the  vector  x  by  the  vector  D.  With  these  definitions  we  can  introduce  the  interpolation 
problem  determined  by  A  and  T: 

Find  p  €  V(A)  satisfying  (16)  for  all  v  €  V(A,T). 

The  solvability  of  (17)  is  due  to  the  following  result  from  Dyn  and  Ron  (1988): 


(17) 


Theorem  1  The  spaces  V(A)  and  the  space 


I 


I 


H(A,T)=  ©  {e-*g(x)|9€/CM.)},  (18) 

*€V(A,T) 

are  dual  to  each  other  under  the  pairing 

[p(£>)/»j(0)=  [q(Z>)p](v)  ,  peV(A),  h(x)  =  e**q(x)  6  H(AJ)  .  (19) 


Corollary  1  There  exists  a  unique  p  €  V(A)  solving  the  interpolation  problem  (17). 


It  follows  from  Theorem  1  that  fC(Ar)  is  dual  to  V(Ar)  in  the  sense  of  (19),  and  by 
(b)  dim  fC(Ar)  =  #  of  bases  in  Ar.  Furthermore,  since  V(Av)  C  we  conclude 

that  fC(Ar)  c  Hence 


^dv-i  C  AC(i4v)  C  |-»  i  (20) 

in  analogy  to  (5).  Moreover,  if  the  directions  in  A T  are  in  general  position  then  AC(AT)  = 


Corollary  2  Let  T  be  such  that  for  each  v  €  V(A,  T)  the  directions  in  are  in  general 
position.  Then  the  interpolation  conditions  in  (16)  are  pure  Hermite  of  the  form 

M 

Dmp(y)  =  Dmf(y) ,  |m|  =  ^Tro,-  < |AT| -a  ,  m<  >  0  ,  »  =  l,...,a  .  (21) 

•=i 


In  case  A  consists  of  directions  in  general  position,  then  so  does  each  A*,  v  € 
V(A,r),  and  the  interpolation  problem  becomes:  Find  p  €  *„-»  satisfying  (21)  for  each 
v  €  V(A,  T).  In  R *  the  conditions  on  T  in  Corollary  2  are  satisfied  if  7*  ^  Xjj  whenever 
**  =  Aa^,  A  6  JR,  »  £  j,  i,j  e  {l,...,n},  namely  if  the  hyperplanes  in  (6) 

are  pairwise  disjoint. 

An  especially  interesting  interpolation  problem  is  the  Lagrange  interpolation,  ob¬ 
tained  when  (A,T)  is  simple.  In  this  case  |A»|  =  s,  £(AT)  =  jr0,  and  p  satisfies 
P(v)  =  /(v)»  v  €  V(A,  T).  The  solution  is  given  explicitly,  in  terms  of  the  basis  (10),  as 

p(x)  =  £  /(v)Pt(i)  .  (22) 

t€V(X,T) 


1 


This  together  with  Remark  1  implies  that  the  interpolation  problem  (17)  is  a  limit  of  a 
sequence  of  Lagrange  interpolation  problems. 

For  general  (A,  T)  the  interpolation  conditions  (16)  at  v  6  T  (A , T )  are  determined 
by  the  structure  of  a  chosen  basis  of  AC(AT).  The  construction  of  such  bases  is  discussed 
by  Dahmen  (this  volume)  and  by  deBoor  and  Ron  (1988). 

3.  Examples 

The  first  two  examples  are  in  R 2  and  can  be  displayed  graphically.  We  consider 
two  Lagrange  interpolation  problems,  for  the  same  set  of  directions  A,  and  then  two 
Hermite-type  problems,  obtained  as  limits  of  the  Lagrange  problems. 

Example  1  Let  A  =  {a1,..., a®}  with  a1  =  a4  =  (1,0),  aJ  =  a5  =  (0,1),  a3  =  a*  = 
(1, 1),  and  let  F  =  (0,0, 1,  j  +  £,  j  +  £,  5)  for  e  >  0.  The  space  V(A)  is  of  dimension 
12  and  consists  of  quartic  polynomials  which  reduce  to  cubics  along  hyperplanes  of  the 
form  a*  •  x  =  const.  The  hyperplanes  a*  •  x  =  7*,  i  =  1,. . .  ,6  are  depicted  in  Figure  1, 
together  with  the  twelve  interpolation  points.  Since  each  interpolation  point  belongs  to 
exactly  two  hyperplanes,  (A,  T)  is  simple,  and  the  data  at  each  point  is  just  the  function 
value. 

For  £  =  0  the  three  interpolation  points  vl  =  (1,0),  v2  =  (0, 1),  v3  =  (0,0)  remain 
unchanged  together  with  the  corresponding  AT< .  Hence  also  in  this  problem  only  function 
values  are  required  at  v‘,  i  =  1,2,3.  Each  of  the  other  three  interpolation  points  v4  = 
(j,0),  v*  =  (0,  j),  v®  =  (y,j)  is  the  limit  of  three  interpolation  points  in  the  case 
e  >  0,  with  Ari  =  {a1, a2, a3},  i  =  4,5,6.  Thus  K(Ari)  =  Tj,  i  =  4,5,6,  and  the 
Hermite  conditions  are  of  the  form 

(/  -  p)(v<)  =  0  ,  JL(f-p)(vi)  =  0  ,  A(/_p)(v-)  =  0  ,  i  =  4,5,6  - 

Example  2  Let  A  be  as  in  Example  1  and  let  T  =  (0,0,  l,£,f,  1  -  e)  for  £  >  0.  The 
space  V(A)  is  as  in  Example  1.  The  hyperplanes  a*  •  x  =  7 i,  t  =  1, ...  ,6  are  depicted  in 
Figure  2.  These  hyperplanes  have  twelve  intersection  points,  each  belonging  to  exactly 
two  hyperplanes.  Thus  for  £  >  0,  (A,T)  is  simple  and  the  interpolation  is  of  Lagrange 
type. 

In  the  limit  £  -*  0,  there  are  only  three  interpolation  points:  v1  =  (1,0),  v2  =  (0,1), 
v3  =  (0,0),  each  being  the  limit  of  four  interpolation  points  in  the  case  £  >  0.  The 


.(!♦*,  z+E) 


Figure  1  Figure  2 

interpolation  conditions  at  v‘  are  determined  by  fC(Avi )  where 

=  {aJ, a^+3  |  j  j*  i  ,  j  =  1,2,3}  . 

By  (15) 

tC(Avt)  =  {p€  t  |  (aJ  •  D)*p  =  0  ,  j^i  ,  j  -  1,2,3} 

3 

=  span{l,ii,x2,  JJ(nJ  '  x)}  > 

i»  l 
i*« 

where  n*  •  a*  =  0,  i  =  1,2,3.  Hence  the  interpolation  conditions  are 

(/  ~  p)(v*)  =  0  ,  A(/_p)(vi)  =  0,  ^-(/-P)(V<)  =  0,  *  =  1,2,3 

5-2L_(/-pMv1)  =  0. 

This  interpolation  problem  is  a  special  case  of  the  one  solved  by  Gregory  (1985), 
where  the  interpolation  points  are  the  vertices  of  a  simplex  in  Bt*,  and  A  consists  of  s  +  1 
directions  in  general  position  each  repeated  N  >  2  times.  The  next  example  deals  with 
an  extended  version  of  this  case  in  terms  of  our  analysis. 


F. yam nl<>  3  Let  B  =  {b1, . . .  ,b*+l}  C  1R*  be  in  general  position  and  let  (B,  A)  be 
simple,  with  A  =  ($i...  .,6,+i)  €  iR*+1.  Given  s  +  1  positive  integers  mi,...,m»+i, 
n  =  m<’  consider  A  =  {a1,..., a"}  consisting  of  b*  repeated  m<  times,  and  T  = 


(7i, •  •  •  ,7n)  consisting  of  6{  repeated  mi  times,  t  =  l,...,s  +  1.  The  hyperplanes  Hi  = 
{x  |  b*  •  x  =  £,},  i  =  1, . . . ,  3  +  1,  intersect  at  s  +  1  points  v1,. . .  ,v*+1,  forming  the 
vertices  of  a  simplex.  Let  v*  denote  the  intersection  of  the  a  hyperplanes  Hj,  j  ^  i, 
j  =  1, . . . ,  s  +  1.  Then  consists  of  bJ  repeated  m3-  times  j  £  t,  j  =  1, . . . ,  a  +  1,  and 
by  (15) 

K{Ari)  =  {p  €  *  |  (b*  •  D)m*p  =  0  ,  j  ^  i  ,  j  =  +  1}  . 

The  dimension  of  K.(A^t)  is  the  number  of  bases  in  Avi  given  by  Af<  =  IJy+i,#. mj- 
Now  the  edge  of  the  simplex  connecting  vertices  v*  and  v*  belongs  to  the  intersection 
of  the  hyperplanes  Hj,  j  £  t,/,  j  =  l,...,s  +  1.  Hence  (v*  -  v*)  •  bJ  =  0,  j  i,l , 
j  =  1, . . . ,  a  +  1,  from  which  we  conclude  that  any  polynomial  of  the  form 

»+i 

[(v<- v*).x]01  ,  0<  at  <  mt  ,  1  =  l,...,s+  1  , 

<■1 

t+i 

is  annihilated  by  (bJ  •  D)m/ ,  j  ^  i,  and  therefore  belongs  to  K(Awi ).  The  number  of  these 
polynomials  is  Jlf<  and  they  are  linearly  independent,  thus  forming  a  basis  of  /C(AT).  In 
terms  of  this  basis  the  Hermite  type  conditions  at  v*  are 
*+l 

II  ((v<  “  vi)  •  D]a>U  -  PKV<)  =  0  i  0  <  aj  <  mj  ,  j  jt  i ,  j  =  1 . s  +  1  . 

y-i 
t+ * 
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ALGORITHMS  FOR  THE  CONSTRUCTION  OF  DATA  DEPENDENT 
TRIANGULATIONS 


NIRA  DYN,  DAVID  LEVIN,  SAMUEL  RIPPA 
School  of  Mathematical  Sciences 
Sackler  Faculty  of  Exact  Sciences 
Tel-Aviv  University 


Abstract  Given  a  set  of  data  points  in  R2  and  corresponding  data  values  it  is  clear  that 
the  quality  of  a  piecewise  linear  interpolation  over  triangles  depends  on  the  specific  tri¬ 
angulation  ol  the  data  points.  While  conventional  triangulation  methods  depend  only 
on  the  distribution  of  the  data  points  in  R 2,  we  suggested  in  [1]  the  construction  of 
triangulations  which  depend  on  the  data  values  as  well.  In  this  paper  we  present  and 
compare  some  algorithms  for  the  construction  of  such  data  dependent  triangulations. 
Key  words  :  Triangulation.  Data  dependent  triangulation,  Piecewise  linear  interpolation. 


1  Introduction 

Let  V  =  {u{  =  (x{,y,)  €  R2  ,i  =  l,...,iV}  be  a  set  of  distinct  and  non-collinear  data 
points  and  F  =  ( Fi , . . . ,  F# )  a  ( real )  data  vector.  Suppose  furthermore  that  II  3  V  is  a 
region  with  a  polygonal  boundary  dQ  with  all  vertices  in  V. 

Definition  I  A  set  T  =  {Ti}\  of  non-degenerate,  open,  triangles  is  a  triangulation  of 
ft  if: 

•  V  is  the  set  of  all  vertices  of  triangles  in  T. 

•  Every  edge  of  a  triangle  in  T  contains  only  two  points  from  V,  namely  its  endpoints, 

•  n  =  U)=iT,  ,  r,nr;  =  0, 


IT 


Given  a  triangulation  T  of  Cl  we  consider  the  space  Si(T)  of  piecewise  linear  poly¬ 
nomials  defined  over  T.  i.e. 

s?(r)  =  {S€C°(fl)|  <7ir,  e  n,}. 

where  II i  is  the  three  dimensional  space  of  linear  polynomials.  Finally  we  denote  by  Jj 
the  unique  function  from  S'i(T)  interpolating  F,  i.e. 

fr(xi,yi)  =  Fi  ,  »= 

A  linear  function  is  uniquely  defined  by  its  values  at  the  three  vertices  of  a  triangle 
and  thus  a  Piecewise  Linear  Interpolating  Surface  (PLIS)  is  uniquely  determined  by  the 
choice  of  a  specific  triangulation  of  D.  It  is  clear  that  the  quality  of  approximation  by 
a  PLIS  depends  on  the  particular  choice  of  the  triangulation  and  naturally  we  look  for 
an  optimal  triangulation.  The  classical  theory  says  that  long,  thin,  triangles  should  be 
avoided  and  that  triangles  should  be  as  equiangular  as  possible  ([4]).  A  popular  choice 
of  triangulation  for  interpolation  schemes  is  the  well  known  Delaunay  triangulation  (see 
e.g.  [4])  which,  among  other  nice  properties,  is  a  MaxMin  triangulation,  i.e.  it  is  a 
triangulation  Tm  maximizing  the  quantity 

a(T)  -  m|n(  smallest  angle  in  Ti). 

The  Delaunay  triangulation,  as  various  others  in  use,  depends  only  on  the  set  V 
and  not  on  the  data  vector  F.  In  [1]  we  suggested  the  use  of  data  dependent  criteria 
for  measuring  the  quality  of  a  triangulation.  These  criteria  depend  on  the  set  V  of 
data  points  and  on  the  data  vector  F  as  well.  Given  a  data  dependent  criterion  and 
an  initial  triangulation,  a  data  dependent  triangulation  T'  may  be  constructed  by  the 
familiar  procedure  of  swapping  diagonals  of  convex  quadrilaterals  in  order  to  get  better 
triangulations  where  "better”  should  be  interpreted  as  better  with  respect  to  the  given 
data  dependent  criterion.  Numerical  tests,  reported  in  [lj,  demonstrate  very  clearly 
that  PLISes  defined  over  data  dependent  tri angulations  provide  better  approximation, 
to  various  test  functions,  than  the  PLIS  defined  over  the  Delaunay  triangulation  of  the 
same  set  of  data  points. 

In  the  present  paper  we  discuss  algorithms  for  constructing  data  dependent  trian- 
gulations.  In  §2  the  concept  of  data  dependent  triangulations  is  presented  as  well  as 
two  algorithms  for  their  construction.  In  order  to  simplify  the  presentation  we  use  an 
example  rather  than  a  more  detailed  definition  to  illustrate  the  ideas  involved.  It  is 
straightforward  to  extend  the  results  to  the  more  general  setting  of  [1].  The  most  basic 


algorithm  for  the  construction  of  data  dependent  triangulations  is  Lawson's  LOP  algo¬ 
rithm  which  swaps  the  diagonals  of  convex  quadrilaterals  in  order  to  decrease  a  certain 
cost  function.  The  Modified  LOP  (MLOP)  suggested  here  defines  the  specific  order  of 
swapping  edges.  Two  strategies  for  swapping  edges  are  presented  in  §3  and  numerical 
experiments  comparing  the  LOP  and  MLOP  algorithms  are  reviewed  in  §4. 

2  Data  dependent  triangulations 

Let  V  be  a  fixed  set  of  data  points,  F  a  data  vector,  T  a  triangulation  of  SI  and  suppose 
that  fx  is  the  piecewise  linear  interpolant  to  F.  For  each  interior  edge  eofT  a  real  cost 
function  5(/r,  e)  is  assigned,  and  the  index  vector  Nx,  of  length  q,  containing  the  cost 
functions  of  all  interior  edges  is  constructed: 

,\’t  =  (S(/r,ei),  —  S(/r,eq)). 

The  cost  function  of  a  triangulation  is  defined  to  be: 

c(/t)  =  51  |S(/t, enl¬ 
ist 

Two  examples  for  cost  functions  are  :  (a)  Jump  in  Normal  Derivative  (JND)  S\(fx,  e), 
the  (magnitude  of  the)  jump  in  the  normal  derivative  of  fx  across  the  edge  e  and  (b) 
Angle  Between  Normals  (ABN)  S2(fx,e),  the  angle  between  the  normal  vectors  to  the 
two  facets  of  the  surface  fx  on  both  sides  of  the  edge  e.  These  cost  functions,  and  others, 
are  discussed  in  more  detail  in  [l]. 

Definition  2  A  triangulalion  T'  of  SI  is  called  optimal  if  c(/j<)  <  c{fj)  for  every  tri¬ 
angulation  T  of  SI. 

In  most  practical  situations  it  is  very  difficult  to  obtain  a  globally  optimal  triangula¬ 
tion  so  here  we  consider  only  locally  optimal  triangulations.  Let  T  be  a  triangulation  of 
SI,  e  an  interior  edge  of  T  and  Q  a  quadrilateral  formed  from  the  two  triangles  having  e  as 
a  common  edge.  If  Q  is  strictly  convex  then  there  are  two  possible  ways  of  triangulating 
Q  (see  Figure  1  ). 

Definition  3  An  edge  e  is  called  locally  optimal  if  one  of  the  following  conditions  holds: 
(a)  Q  is  not  strictly  convex  or  (b)  Q  is  strictly  convex  and  c(fx)  <  c(fx')  where  T'  is 
obtained  from  T  by  replacing  e  by  the  other  diagonal  of  Q. 


Figure  1:  Two  triangulations  of  a  convex  quadrilateral 


Definition  4  A  locally  optimal  triangulation  of  fi  is  a  triangulation  V  in  which  all 
edges  are  locally  optimal. 

The  basic  algorithm  for  constructing  locally  optimal  triangulations  is  the  Local  Op¬ 
timization  Procedure  (LOP)  suggested  by  Lawson  [3]: 

Algorithm  1  LOP 

1.  Construct  an  initial  triangulation  of  ft  and  set  T  —  T(0L 

2.  If  T  is  locally  optimal,  end  the  procedure;  else  go  to  step  3. 

3.  Let  e  be  an  interior  edge  of  T  which  is  not  locally  optimal  and  let  Q  be  the  strictly 
convex  quadrilateral  formed  from  the  two  triangles  in  T  having  e  as  a  common 
edge. 

(a)  Swap  diagonals  of  Q:  Replace  e  by  the  other  diagonal  of  Q,  therefore  trans¬ 
forming  T  into  a  triangulation  T' 

(b)  Set  T  —  T'  and  go  to  step  2. 

Each  time  an  edge  swap  occurs,  the  cost  function  of  the  resulting  triangulation  is  strictly 
smaller  than  that  of  the  previous  one.  Since  the  number  of  triangulations  of  ft  is  finite, 
the  LOP  converges,  after  a  finite  number  of  edge  swaps,  to  a  locally  optimal  triangulation. 

The  above  LOP  seems  to  work  very  well  in  the  numerical  experiments  reported  in 
[1],  but  it  has  a  major  drawback:  the  resulting  locally  optimal  triangulation  depends 
on  the  labelling  of  the  data  points  and  on  the  software  implementation  of  the  LOP. 
We  would  like  to  control  more  closely  the  order  in  which  edges  are  swapped  during  the 
LOP  iterations  in  order  to  obtain  a  better  defined  algorithm.  Given  a  triangulation  T 


we  consider  the  set  E(T)  of  all  the  interior  edges  c  of  T  which  are  not  locally  optimal. 
Suppose  that  E(T)  ^  0  and  assume  that  the  edges  are  labeled  such  that: 


E(T )  —  {Cl,  ^2, - ek{T)}- 

A  swap  of  some  edge  ej  ,  1  <  j  <  k(T),  will  transform  T  into  a  triangulation  The 
Modified  LOP  (MLOP)  selects,  at  each  step  from  the  set  E{T)  .  the  next  edge  to  be 
swapped  according  to  a  predetermined  swapping  strategy.  In  §3  we  discuss  and  compare 
some  selection  strategies.  Suppose  that  the  edge  ep  €  E(T )  is  swapped.  Then  J E'(T(p) ) 
has  to  be  computed  for  the  next  iteration.  This  can  be  done  efficiently  since  E(T^) 
and  E(T)  differ  only  in  edges  belonging  either  to  the  two  triangles  T,  and  T:  which 
have  ev  as  a  common  edge  or  in  edges  belonging  to  triangles  sharing  an  edge  with  Tt  or 
Tj.  Thus  at  most  12  edges,  which  may  be  in  one  of  the  sets  and  not  in  the  otW,  need 
to  be  checked  in  each  MLOP  iteration  (after  the  edge  swap,  ep  becomes  locally  optimal 
and  thus  is  excluded  from  the  set  E(T M)). 

3  Selecting  the  edges  to  swap 

The  first  selection  strategy  is  the  Maximal  Reduction  (MR)  strategy.  Since  a  data 
dependent  criterion  selects  triangulations  which  minimize  the  cost  function  c(/j-),  a 
natural  strategy  for  the  MLOP  is  to  swap,  in  each  iteration,  the  edge  ep  €  E(T)  for 
which  the  c(fop))  is  minimal,  i.e. 

c(/r<p>)  —  c(/t(j))>  1  <  j  S.  k(T). 

The  second  strategy  is  based  on  the  observation  that  often  the  LOP/MLOP  terminates 
in  a  poor  local  minimum  since  many  of  the  edges  become  interior  to  non-convex  quadri¬ 
laterals.  To  avoid  this  we  would  like  to  swap  the  edges  in  a  way  which  will  leave  the 
maximum  possible  number  of  convex  quadrilaterals  for  the  next  MLOP  iteration.  Let 
us  divide  the  set  E(T)  into  classes  of  edges  according  to  the  value  of 

m}  =  I(T^)-I(T),  j=  1 . k(T), 

where  I(T )  is  the  number  of  convex  quadrilaterals  in  the  triangulation  T.  Let 

E'(T)  C  E(T) 

be  the  set  of  edges  ej  €  E(T )  for  which  is  greatest.  The  Maximal  Opportunity 
(MO)  strategy  chooses  the  next  edge  ep  to  swap  from  the  set  E  (T)  such  that  c(fT{r, )) 


is  minimal,  i.e. 


c(f-np))  <  c(/7u>),  e}  €  E(T). 

For  comparison  we  also  generate  the  "worst  possible’’  sequence  of  edge  swaps.  This 
is  obtained  by  the  Minimal  Reduction  ( MinR )  strategy  which  is  just  the  opposite  of  the 
Maximal  Reduction  strategy,  i.e.  we  choose  the  next  edge  to  swap  so  that  the  reduction 
in  the  value  of  the  cost  function  is  minimized. 

4  Numerical  experiments 

In  our  numerical  experiments  two  sets  of  data  points  taken  from  Franke  ([2])  were  used. 
The  first  set  contains  100  data  points  distributed  more  or  less  uniformly  over  the  unit 
square,  while  the  second  set,  with  33  data  points,  was  designed  with  larger  variations 
in  the  density  of  the  data  points.  The  data  vectors  F  =  (Fi,...,Fv)  were  obtained  by 
evaluating  various  test  functions,  also  used  in  [1],  at  the  data  points. 

Several  data  dependent  criteria  were  tested  including  those  relating  to  the  JND  and 
ABN  cost  functions  mentioned  in  §2  as  well  as  others  presented  in  [1].  For  each  data  set 
and  data  dependent  criterion,  several  data  dependent  triangulations  were  generated  by 
using  different  strategies  for  swapping  the  edges: 

•  LOP  -  No  strategy,  edges  are  swapped  according  to  their  labelling  in  the  edge  list. 

•  MLOP-MR  -  Edges  are  swapped  according  to  the  Maximal  Reduction  strategy. 

•  MLOP-MO  -  Edges  are  swapped  according  to  the  Maximal  Opportunity  strategy. 

•  MLOP-MinR  -  Edges  are  s.vapped  according  to  the  Minimal  Reduction  strategy. 

The  Delaunay  triangulation  of  the  data  points  was  used  in  all  cases  as  an  initial  trian¬ 
gulation. 

On  each  triangulation,  the  piecewise  linear  function  fj,  interpolating  the  data  vector 
F,  was  constructed,  and  the  error  between  fj  and  the  test  function  which  generates  F 
was  computed  on  a  grid  of  33  x  33  nodes.  The  mean,  root  mean  square  and  maximum 
of  these  errors  were  tabulated  along  with  the  value  of  the  cost  function  and  the  num¬ 
ber  of  edge  swaps  needed  for  convergence  of  the  LOP/MLOP  to  the  data  dependent 
tri  angulation. 

In  the  numerical  experiments  it  became  clear  that  the  order  in  which  edges  are 
swapped  during  LOP/MLOP  iterations  may  have  a  large  influence  on  the  final  locally 
optimal  triangulation  in  terms  of  the  quality  of  approximation  to  the  test  functions  and 
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the  value  of  the  cost  function  of  the  locally  optimal  triangulation.  It  was  found  that  the 
value  of  the  JND  and  ABN  cost  functions  of  a  triangulation  is  usually  a  good  indicator 
to  the  quality  of  approximation  to  the  test  functions:  a  PLIS  defined  over  a  low  cost 
triangulation  (for  which  these  cost  functions  have  small  values)  usually  provides  a  better 
approximation  than  a  PLIS  defined  on  a  high  cost  triangulation. 

The  MLOP  with  the  MR  edge  swapping  strategy  performed  quite  well  in  general.  The 
resulting  data  dependent  triangulations  were,  in  most  cases,  comparable  or  better  than 
data  dependent  triangulations  resulting  from  other  strategies  of  edge  swapping.  When  we 
say  “better”  we  mean  that  a  better  approximation  to  the  test  functions  was  achieved. 
The  MLOP-MR  triangulation  was  in  most  cases  the  triangulation  which  attains  the 
smallest  value  of  the  cost  function.  Also  the  MLOP-MR  converges  in  the  fewest  number 
of  edge  swaps  and  thus  is  the  most  efficient.  In  view  of  these  reasons  this  is  the  scheme 
of  our  choice. 

The  MLOP-MO  strategy  was  comparable  to  the  MLOP-MR  strategy  but  is  more 
difficult  to  program  and  thus  we  do  not  recommend  the  use  of  it. 

The  MLOP  with  the  MinR  strategy  did  produce  in  most  cases  the  worst  data  depen¬ 
dent  triangulation  of  all  and  used  the  largest  number  of  edge  swaps.  The  MLOP-MinR 
triangulation  demonstrates  that  there  is  a  sequence  of  edge  swaps  which  can  lead  to  very 
bad  data  dependent  triangulations. 

As  can  be  expected  from  the  above  observations,  the  LOP  swapping  according  to 
labelling  resulted  in  good  and  bad  triangulations  depending  on  the  labelling  used,  the 
triangulation  criterion  and  the  test  function.  The  results  are  usually  quite  acceptable  but 
often  worse  than  the  MLOP-MR  triangulation  and  sometimes  much  worse.  There  are 
advantages,  however,  in  using  the  LOP  since  its  programming  is  simpler  and  it  requires 
less  computer  storage  than  the  MLOP  for  which  the  list  E(T)  of  edges  has  to  be  stored 
and  maintained. 

We  note  that  none  of  the  above  strategies  performed  well  in  all  cases.  For  any  of 
these  strategies  there  are  examples  of  poor  data  dependent  triangulations  generated  by 
it.  It  may  be  interesting  to  look  for  other  heuristic  strategies  for  edge  swapping. 

An  example  of  data  dependent  triangulation,  taken  from  [1],  concludes  this  paper. 
The  cliff  function  F  =  (tanh(9y  -  9x)  +  l)/9  (see  Figure  2),  was  sampled  on  a  set  of 
33  data  points.  The  Delaunay  triangulation  of  the  set  of  data  points  and  the  related 
PLIS  are  displayed  in  Figure  3.  A  data  dependent  triangulation,  based  on  the  JND  cost 
function  of  §2,  and  the  PLIS  defined  over  it  are  displayed  in  Figure  4.  These  pictures 
demonstrate  very  clearly  the  advantage  in  using  data  dependent  triangulations. 


Figure  4:  A  data  dependent  triangulation  and  the  PLIS  defined  over  it 
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ALGORITHMS  FOR  COMPUTING  BEST  PARAMETRIC  CUBIC  INTERPOLATION 
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Abstract  We  consider  the  problem  of  finding  a  curve  passing  through 
prescribed  points  in  such  that  the  kinetic  energy  of  particle 
moving  along  this  curve  is  minimized.  Recently  uniqueness  of  the 
solution  of  this  problem  has  been  establ ished  by  the  second  author. 
In  this  contribution  algorithms  for  computing  the  solution  are 
devised  and  their  (global)  convergence  properties  studied. 

Key  words:  Best  interpolation,  cubic  spline  curves,  constrained 
minimization,  stationary  points,  projected  gradient  method,  Newton 
Method. 


1.  Introduction.  The  problem  of  best  parametric  interpolation  of 

given  data  in  Rd  is  the  following:  find  a  function 

^  i  js(t)  6  Jf~(0,l),  the  space  of  R^-valued  functions  on  [0,1]  with 
‘  1  r  k 

3  components  in  l_2(a»b)>  which  attains 

4 

6  i  inf  inf  {/  lif(k)(t)l  |2dt  :  f  €  £*(0,  l),f_(t.)«y.  ,  1  <i<n}.  (1) 

7  t_  o 

8 

9  The  first  infimum  runs  over  all  sequences  t  of  nodes  satisfying 


0-t.  <  t_  <  ...  <t 
1  2  n 


and  the  norm  II  II  in  (1)  denotes  the  usual  Euclidean  norm  in  ®.  .  If 
one  considers  only  tne  second  infimum  in  (1)  the  problem  reduces  to 
the  classical  problem  of  best  interpolation  in  spline  theory.  For 
the  motivation  of  the  general  problem  we  refer  to  [5]  ,  [3]  and  [4]. 
In  [3]  and  [4]  the  uniqueness  of  the  solution  of  (1)  in  the 
cubic  case  k  =  2  is  proved  (d  =  1  in  [3]  and  general  d  in  [4])  under 
the  condition 


*i  *^i+l  ’ 


1  <  i  <  n- 1  , 


on  the  data. 


In  this  note  we  describe  two  types  of  convergent  algorithm  for 
computing  the  solution  s*(t)  of  (1)  in  the  case  k  =  2.  First  we 
summarize  some  well-known  facts  and  the  relevant  results  of  [4].  For 
fixed  t  the  inner  infimum  in  (1)  is  attained  by  a  cubic  spline 
function  s(t)  whose  second  derivative  is 


s"(t)  -  1  a.  N.  ,(t), 

i=l  1  1,Z 


where  the  N.  9(t)  denote  the  piecewise  linear  B-splines  with  knots 
1  ,  Z 

in  t  satisfying 


N.  _(t.)  =  6.  .  .  , 
i.2  2  J.i+l 


1  <  j  ,  i  <  n-2 


The  coefficients  F.  in  (4)  are  therefore  equal  to  s"(t^+)) 
They  are  determined  by  the  tridiagonal  linear  system 

n-2  w .  w. 

I  G .  .  a .  »  z  .  :«  - t-J-  ,  w.  :»  y .  -  v .  ,  (5 

i-,  1J  _1  Vi  hj  -j  xj+1 


where 


2 (h.  +  h.  + , )  ,  j-i. 

hmax(i,j)  ,  li-jl=l  ,  »  (6) 


otherwise • 


to  IT*  LP  to 


I 


We  can  write  (5)  in  matrix  form  as 

G  A  =  Z  (7) 

if  we  introduce  the  vectors 

-  5"  (— l  *  •  •  *  *^n-2 }  . 

Furthermore  one  can  easily  verify 
1  2 

/  I ls"(t) I  I  dt  =  <  A  ,  GA  >  *  <  A  ,  Z  >  ,  (8) 

o  ~ 

where  <  ,  >  denotes  the  obvious  scalar  product.  From  this  one  con¬ 
cludes  that  the  infimum  problem  (1)  is  equivalent  to 


inf  { F  (h)  :  F(h)  :-<A,Z>,  GA  =  Z>, 
h€K 


where 

K 


(h  €K  :  h  -  (hj , . . .  ,h  )  ,  lu  >  0,  Ih.=  ]}. 

i= ) 


Hence  a  solution  of  (1)  is  given  by  a  pair  (h  ,  A(h) ),  where  A»A(h)  , 
satisfies  (7)  and  h  the  equations  for  a  critical  point  of  F(h)  in  K. 
These  equations  read 


i  .  3F  _  , 

2T . 

,i 

R 

> 

J 

n-l 

I  h., 
j"l  J 

1  <  j  <n-l  , 


with  Lagrangian  parameter  A  €R.  For  the  quantities  in  (11)  one  has 

the  explicit  expressions  (with  a  -  a  .  ■  0) 

-o  — n-1 


(Ha) 

14 

13 

(lib)  12 

,  11 

i  in 


T.  -  h?  <  A  ,  -^r 

J  J  —  3h. 


HT>m  (^j-l  "VV* 


(12a) 
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01  j  <  -  *  ShT  -  >  =  3  t  (ij-lj)  +  +  (ij.lj-j)  ]•  02b) 

Here  (  ,  )  denotes  the  usual  scalar  product  in  Rd. 

The  equations  (11)  together  with  (7)  form  a  nonlinear  system  of 
equations  for  h  £  K.  In  [4]  the  following  was  proved: 


Theorem  1 :  There  exists  only  one  minimum  h*  of  F(h)  in  K  (which  ' 

is  then  the  unique,  global,  solution  of  (1)).  The  corresponding  i  4 

pair  h*  ,  A*  “A(h*)  is  a  solution  of  the  saddle  point  problem  4 1 

1 

inf  sup  <p  (A,h)  ■  sup  inf  $  (A,h), 

hOC  A  A  hOC  :  3 1 

where  <p  (A,h)  :-  -  <  A,GA  >  +  2  <  A,Z  >.  A  critical  point  of  F(h), 

1. e.  a  solution  of  (II)  and  (7),  is  a  solution  of  (1)  if  and  only 
if  the  corresponding  A*  lies  in 

2 1 

B  {A€R(n2)d  :  T^  :*  (a^-a.,^)  >0,  l<j<n-l}.  (14) 

2  ( 

2.  Descent  Methods.  In  the  following  we  describe  how  some  descent 
methods  for  constrained  minimization  problems  may  be  adapted  and 
modified  to  problem  (9)  so  as  to  guarantee  global  convergence.  A 

") 

general  method  for  minimization  with  linear  equality  constraints  has  ■> 
the  form:  1 

Given  the  approximation  h^  for  the  solution  h*  €  Rn_1  ,  one 
computes  an  (improved)  approximation  h^v_1 ^  for  h*  by  if 


h(v+,)  :-  h(v)  +  6  Pd(v) 


v  -  0,1,...,  (15) 


where  is  the  direction  of  search  (lld^ll  ”1),  the  positive 

scalar  6^  the  step  length,  and  P  the  linear  projection  of  d^  to 
ker  R,  where  the  constraints  are  given  in  the  form  Rh  -  b  with 
matrix  R.  In  the  case  of  problem  (9)  a  simple  choice  is  to  set 


(P  d ) .  d.  _  1  I  d. 

_ 3  J  (n-l)  1 


ro  eg 
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I 


for  the  vector  d  =  (d ,,..., d  ,). 

—  l  n-  i 

In  order  that  the  above  method  is  a  descent  method  we  assume 

that  there  is  a  number  6  >  0,  independent  of  v  *  0,1,2, _ ,  such 

T 

that  (note  that  P  =  P  with  the  choice  (16)) 


i.  i 

1  3 

14 


(sW,Fd(,))  <-  BIIpV'’)II, 

(v)  ,  (v) 

where  g  denotes  the  gradient  of  F  at  h  ,  i.e. 

£<V)  grad  *(h(v)). 


(17) 


n-1 


(Note  that  here  (,)  stands  for  the  scalar  product  in®  .)  Then 
Taylor  expansion  gives  the  decrease  of  F(h^v+1^)  for  6^  sufficiently 
small.  The  following  step— length  procedure  realizes  this. 

Choose  6^  *  2  J ,  where  j  =  j (v)  is  the  smallest  non-negative 
number  such  that 

F^h(v)  +2_jd(v))  <  F^h^V^  y  +  2-^_l  ^(Nj),Pd  (V)^  (18a) 

is  valid.  In  this  procedure  the  constraints  0<hfV+1^  <1,  1  <i<n-l, 
implied  by  h^  6K  can  also  be  incorporated.  To  this  end  we  require 
additionally  that  6  =»  2  J  shall  satisfy 
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(18b) 
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Both  (18a)  and  (18b)  are  always  satisfied  for  =  2  J  sufficiently  r 

small.  This  is  guaranteed  by  the  fact  that  for  any  h£K  the  inequality  l 

'  ;  1 

,-y. I |2+F(h)]  (19)  ,  10 

9 

8 


min  h.  >  min  I ly. . ,  -  y. I  I  /  [(n-l)maxl ly.  -y. I 

.  j  •  j  -j+1  O 


holds  (see  [4]). 

The  above  considerations  only  ensure  convergence  to  a  critical 
point.  In  order  to  improve  this  we  introduce  so  called  "T  steps" 
for  h^  GK  such  that  not  all  corresponding  T. 's  defind  by  (1 2a) 
are  positive.  In  this  case  a  critical  point  H*  of  F(h)  cannot  be  the 


to  LaJ  .1*  U1  <Ts  J  CO  O  O  h-*  t  >  t*)  lot 


solution  of  (1)  according  to  (14).  Suppose  it  is  not  a  strict  local 
minimum  too.  Th*.n 


< 


lldll-l 


a  ,  tun.’-;  r  £ j 


n  *  min  (  P  d  ,  K(h*)P  d  )  <  0 


(20) 


must  hold,where  H(_h )  denotes  the  Hessian  of  F(h)  at  a  point  h.  If  d* 

is  a  direction  attaining  n  in  (20), the  decrease  of  F(h)  along  cl* 

satisfies,  for  0  <  <5  <  s S  , 

o 


F (h*  +  6Pd*)  <  F (h*)  +  max  (-C6  °,  ri62) 


(21) 


with  some  constant  C  >  0  and  a  natural  number  k  .  This  can  be  seen 

o 


from  Taylor  expansion  about  h*. 

Therefore  we  define  a  "T-step"  as  follows:  for  an  estimate  h^ 
compute  a  direction  d  for  which 


(v) 


n<v,.(pd(',),H(h<v))Pd(v)) 


min  ^Pd  ,  H(h^)Pdj. 


Id  I  I 


The  step  length  6  for  this  direction  is  chosen  as  6  *  2 


,~£o 


V(2  °)  *  min  (tp(2  *■)  :  cp(2  ~)  <  F(h 


(v) 


+  6  Pd 

v 


($ 


)}. 


(22) 


and  &  by 
(18c) 


Here  each  j  has  to  satisfy  (18b),  and  ip(t)  is  defined  by 

<p(t)  :*  F(h^  +  tfdvV^).  We  remark  that  (18c)  can  be  satisfied 

only  if  some  T^  are  non-positive.  Therefore  we  combine  the  "T-steps" 

with  the  "normal  steps"  to  obtain  the  following  algorithm: 

Given  an  estimate  h^  for  the  solution  h,  compute 


the  vector  A ^  *  A(h^)  from  (7)  with  G»G(h^v2),  Z=Z(h^), 


(v) 


(v). 


(v). 


(v)  .  .  .  (v) 

ii)  a  direction  of  search  d,  and  its  projection  Pd  according 

to  (16)  such  that  (17)  is  satisfied,  and  a  step  length  c?v 

according  to  (18a),  (18b), 

(iii)  the  new  estimate  h^v+*^  via  formula  (15),  if  all  T.  formed  in 

/  \ 

(12a)  with  respect  to  A'v  are  positive,  otherwise 


!  1 


iv)  a  direction  of  search  dv  according  to  (22),  and  the  test 

(18c), 

.  ( v+ 1  )  .  (v+l  )  (v)  ^  ~(v) 

v)  the  new  estimate  h  by  setting  :=  h  +  a^d 

if  the  test  is  positive,  otherwise  define  h^V+'^  as  in  iii). 


We  can  now  prove 

Theorem  2:  The  above  algorithm  converges  under  the  assumotion  (18) 
for  any  starting  vector  h^°^  £ K  to  a  strict  local  minimum  of  problem 
(9). 

•  (  \)) 

Proof  By  (19)  all  the  iterates  h  lie  in  a  compact  subset  of  K. 
Hence  there  is  a  subsequence  converging  to  some  h*  £  [0,1]°  which 

/  -i  oo 

we  denote  by  {h  J 

J  —  v=l 

Following  standard  lines  (cf.  [2])it  can  be  established  that  h* 

is  a  critical  point  of  F(h).  But  we  have  still  to  show  that  h*  is  a 

strict  local  minimum  of  (9).  To  this  end  we  choose  for  e  >  0  suitable 

v  =  v  (e)  so  large  that  for  all  e>v 
o  o  o 


I  lh(jv)  -h*l  I  <e,  l||H(h^v))  -  H(h*) 


<Y,U),  v>Vo, 


where 


denotes  the  matrix  norm  on  F  and  Yj  (e)  is  a  constant 


tending  to  zero  for  €->-0.  Then  we  can  find  y2(e)  with  y 2(e)->-0  for 

e -► 0  such  that,  for  v>v  , 

’  o  ’ 


F^h(jv)  +  SPd^v)j  <  F^h*+SPd*)  +Y2(e), 


where  d*  is  defined  via  (20).  This  is  a  consequence  of  the  fact  that 
n  and  d*  depend  continously  on  h*  . 

We  assume  then  that  h*  is  not  a  strict  local  minimum  of  (9),  i.e. 
(20)  holds.  Then  it  follows  from  (23)  and  (21)  (for  0jv<6Q)  that 

<p(2~l)  »F^h(jv)  +  2~*d(jv))  <F(h*)  +max^-C2  °,n4"^  +  Y2(e) 


3  0  0;)  -.n  ^  W  l.)  H  O  \D  CO 


<  F^h^v)  +  c^vPd(jv)j  +  max^-C2  0  ,n  4~^j  +  y2(e)- 


.3 

14 


This  means  that  in  (18c)  a  finite  step  length  a.  =2  <6/2  will 

jv  o 

be  chosen  provided  e  -  and  hence  v  =  v  (e)  -  are  chosen  such  that 

o  o 

Y„(e)  (l /2)min(C,~n) •  Then  we  obtain  a  contradiction  since  £  is  such: 

o 

that  _£ 

F  (]l(jV)  +  1  )  * <P(2  °)<F(h*)  +  (l/2)max(-c  ,  n). 

As  an  application  of  this  theorem  we  state 

Corollary:  The  above  algorithm  converges  to  a  strict  local  minimum 

(v) 

if  one  takes  as  direction  of  search  d  the  direction  of  the  pro¬ 


jected  gradient  Pg 


(v) 
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3.  Further  methods.  The  proposed  modification  of  the  projected 
gradient  method  is  not  yet  practical,  since  a"T  step"  requires  too 
much  work.  This  is  due  to  the  fact  that  the  Hessian  of  F(h)  is  given 
only  implicitly,  namely  by  (cf.Ul) 
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3  F 


3h.3h. 
i  J 


<  ,  G  >  +  6.  .  T.hT3 

3h.  3h.  11  l  l 

i  J 

,n-l 


where  the  vectors  {3A/3h^K  =  j  have  to  be  determined  by  the  systems 
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G(3A/3h.)  =  (3Z/3h.)  -  (3G/3h.)A. 


(24) 


Thus  the  amount  of  work  of  a  "T  step"  is  n-times  larger  than  that 
of  a  usual  one  if  one  assumes  that  the  essential  work  is  solving  the 
(n-2)  *  (n-2)  tridiagonal  systems  (with  always  the  same  matrix  G). 
Namely, a  "normal"step  requires  the  solution  of  d  such  systems 

(see  (5),  (9)  for  the  evaluation  F(h)) whereas  a  "T  step"  needs 
(n-l)d  additional  solutions. 

In  the  actual  computation  we  used  a  simplified  version  of  a 


"T  step"  based  on  the  use  of  directions  e 
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defined  via 


co  o  h  tJ  i.j  ui  a>  -  j  coo 
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-  1.  i  -  t, 
1.  i  -  ~  l. 


otherwise , 


•  •  •  •  ^*(v)  .  , 

and  the  following  definition  of  d  in  the  above  algorithm: 


d(v)  :=■  L  e('£)  ,  I(v)  :=  U  6  {2, . . .  ,n-l }  :  T^v)  <  0,  (25) 

£ei<v>  (v) 

!  where  the  T^^  are  defined  via  (12a)  with  respect  to  A^a  A(h^v^) . 

The  reason  for  the  restriction  ££I^V^  is  that  in  the  first  order 


(VK 

expansion  (h^  sh^  J 


p(h<")*«J(''))»F(h(''))  ♦  «  I  [<-f)-o‘:!*2l'v)h-2-2T|!>h-; 

£erv) 

(v)  (v) 

the  negative  part  with  the  T»  .  T^_j  should  dominate. 


We  used  direction  (25)  combined  with  the  step  length  procedure 
(18b),  (18c)  in  the  above  algorithm  successfully  despite  the  fact 
that  we  could  not  prove  that  it  is  a  direction  of  descent  near  a 
critical  point  other  than  the  solution  of  (1).  Our  experience  was 
that  it  usually  produced  a  larger  decrease  of  F(h)  than  the  (projected) 
direction  of  steepest  descent.  The  effectivness  of  this  technique  is 
highlighted  by  the  fact  that  only  with  its  help  the  solution  of  (1) 
could  be  found  in  presence  of  another  critical  point  (see  example  2 
below).  Here  one  should  remark  that  this  statement  is  not  quite  precis# 
in  that  even  Theorem  2  only  guarantees  convergence  to  a  strict  local 
I  minimum  of  F(h).  However  it  is  very  likely  (with  homotypy  arguments) 
that  there  is  only  one  such  minimum,  namely  the  global  one.  i 

i  Another  idea  is  to  use  Newton's  method.  We  omit  the  details  since  j 
;  each  iteration  would  require  the  solution  of  (n-l)d  tridiagonal 
systems  for  the  computation  of  the  Hessian  (cf.  (24)).  Instead  we 
used  a  less  expensive  variant  based  on  a  different  parametrization  of  ! 
F(h)  which  will  be  described  shortly.  Introducing 
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s.  :=  s'(t.),  S  :=  (s,,...,sn). 


we  obtain  after  some  computation, instead  of  (8), the  representation 
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,2-3 


i  r  n 

Jl  |s"(t)  I  Idt  =  12  I  I  1^1  |ihiJ-<  S,R  >  +  <  s,rs  >| 
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where  R  (r,,...,r  )  is  given  by 
—  — 1  — n 


f(h,S),  (26) 
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r.  :=  V 

w. 

hT2  +  w.  hT2  , 

J  -3-1  J-l 

2  <  j  <  n- 1 
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and  T  ■  (I\.)  denotes  the  symmetric  nxn  matrix 
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Since  any  cubic  spline  function  with  £(t..),  £*  (t . )  prescribed  for 
1  <  i  <  n  is  feasible  in  the  infimum  of  (1),  this  problem  is 
equivalent  to 

inf  .  ¥(h,  S) . 

h€K,  S  €» 

The  equations  for  a  critical  point  h  6  K,  S  of  T(h,  S)  read 
_  W  36 ....  2  24 ,  . 

°.p+_:  ,  y  -—Mw.11  ^(Wj.s.^.s.)  - 

j  hj 
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(28b) 


0  =  a  -  1 2R  +  24  r  S  , 

OO  ' 


(28c) 


with  Lagrangian  parameter  p.  The  notation  ZV/ZS  stands  for  the 
gradient  with  respect  to  the  variables  in  _S.  It  is  well  known 
([1,  chapter  4])  that  (28c)  is  a  defining  system  for  the  natural 
cubic  interpolating  spline  curve  in  Hence  the  solutions  of 

equations  (8),  (11)  in  h,  A  are  same  as  those  in  (28j. 

In  order  to  compute  the  Jacobian  of  (28)  the  n  *  d  unknowns  of 
these  n  *  d  equations  are  ordered  as 

V”*,hn-l,hn!"  Ui  -  *  (-l’’"’^d)  ;  -r'^P . • 

The  associated  n*d  mappings  are  (h  denotes  now  the  old  h  enlarged  by 


cp.  (h,  S)  :»  h  +  3'P/3h.,  l<i£n-l, 
l  —  —  n  l 


ip  (h,S)  :»  I  h.  -  1  ,  i  m  n. 


<(..  „(h,S):«  l<i<n,  l<£<d. 

1  j  -C  ""  X 

The  Jacobian  J  of  this  mapping  on  R0<*  into  Fnd  then  has  the  block  form! 
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where  the  numbers  n j  are  given  by 
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n.  =  144l|w.||2h"5-72(w.,s.+s.  ,)hT4  +  8h.3[  lls.l|2  +  Ms.  .11  + 

J  -J  J  -J  -J  -J  +  l  j  J  [_  J  — j+1 

+  •  (30) 

The  only  nonzero  entries  of  the  matrices  are 
b«>  .24.(«hT3  -  S^’hT2  -  4s<«hT2, 

b<*>  ,  24.<«hT3  -  Ss^hT2-  4s^>hT2  ,  1  <  i  <n-l  , 

i,i+l*  ii  l+li  ii  ’ 

and  the  matrix  T  is  defined  by  (27).  With  the  knowledge  of  J  the 
Newton  method  for  solving  the  system  (30)  can  be  set  up  in  the 
standard  manner.  We  omit  the  details  due  to  the  lack  of  space.  We 
only  mention  that  by  using  the  block  structure  of  J  the  solution  of 

the  linear  equations  with  matrix  J  can  be  reduced  to  the  solution  of 

.  d  -l  x 

I  d  systems  with  matrix  F  and  one  system  with  matrix  E-  I  B  T  B  . 

v=l  v  | 

In  the  actual  computation  we  simplified  J  by  setting  the  matrices 
T  1 

Bj,...,Bd  equal  to  zero.  Then  the  solution  of  the  resulting  system 
requires  the  solution  of  d  systems  with  T  and  one  system  with  E. 

Thus  the  amount  of  work  for  computing  a  new  direction  of  search  is 
comparable  to  that  of  the  projected  gradient  method  above.  We  then 
formed  a  descent  method  with  this  direction  of  search  and  step  length 
,  procedure  as  above  together  with  "T  steps"  in  the  version  of  (25).  We 
could  not  prove  global  convergence  of  this  Quasi-Newton  method.  But 

j  we  have  at  least  local  convergence  in  view  of  the  relation 

1 

!  "j  ■  '2Yj3  *  8hj3i"^j"2  *  "ij.i"2  * 

which  follows  from  (30)  and  (12a).  Specifically  in  view  of  (14),  all 
these  numbers  are  strictly  positive  near  the  solution  and  hence  E  has 
a  bounded  inverse. 


4.  Numerical  results.  Many  examples  were  computed  in  order  to 


i 


compare  the  behaviour  of  che  projected  gradient  method  (PG)  with 
that  of  the  Quasi-Newton  method  (QN) .  As  starting  points  for  the 
algorithms  we  chose  either  the  uniform  parametrization  (U)  with 
knots  h^  =  l/(n-l)  or  the  normalized  accumulated  chord  length  para¬ 
metrization  (CL). 

It  was  observed  that  for  the  PG  method  a  start  with  CL  parametri¬ 
zation  often  took  fewer  iterations  for  convergence,  whereas  for  the 
QN  method  this  happened  for  the  U  parametrization.  Concerning  the 
dependence  of  the  data  it  turned  out  that  for  "smooth"  data  (cf.  the 
data  in  S-form  in  [3])  the  QN  method  often  needed  more  iterations 
than  the  PG  method  but  even  fewer  function  evaluations  (which 
correspond  to  the  number  of  solutions  of  tridiagonal  systems).  For 
data  with  isolated  corners  or  peaks  the  QN  method  seems  to  be 
generally  superior.  The  following  table  illustrates  this  ( #  It 

denote  the  number  of  iterations,  #  F  the  number  of  function  evalua- 

-4 

tions,  the  tolerance  for  the  error  was  chosen  as  10  ): 


CL 

U 

Example  2 

QN 

method 

#  It  [  41 

#  F  |  130 

#  It  |  114 

—  —  —  —  —  —  —  — 

#  F  (  674 

PG 

method 

#  It  J  332 

#  F  j  2737 

#  It  J  555 

#  F  J  4481 

The  following  two  examples  (figures  1,  2  and  3)  are  chosen  to 
illustrate  the  qualitative  behaviour  of  the  different  parametric 
spline  interpolants.  The  curved  marked  by  ...  uses  the  uniform, 

-  -  the  CL  and  -  the  "optimal"  parametrization.  The  small  circles 

mark  the  data  points  (in  figures  1  and  2  the  four  lower  corner  points 
are  nearly  identical) . 

The  first  example  is  particularly  interesting  since  it  exhibits 
the  existence  of  a  critical  (or  stationary)  point  other  than  the 
global  minimum.  The  PG  method  without  "T  steps"  terminated  at  a 
point  with  T^'s  alternating  in  sign  (figure  2)  and  at  a  point  with 


positive  if  "T  steps"  were  used  (figure  1).  According  to  (14) 
only  the  latter  point  is  the  global  minimum. 

The  second  example  (figure  3)  illustrates  very  clearly  the 
influence  of  the  parametrization  on  the  resulting  spline  inter- 
polant.  The  data  points  yj,...^^  and  ^33*  •‘■’^69  f°H°w  cl°sely 
each  other  so  that  the  curve  is  well  described  by  them.  Consequently 
the  parametrization  has  nearly  no  effect;  all  three  curves  are 
essentially  identical.  However,  between  y ^  and  y^  there  is  a  large 
gap  in  the  data  which  provide  therefore  only  poor  information.  Here 
the  use  of  sound  parametrization  turns  out  to  be  essential. 
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Abstract:  In  this  paper  we  deal  with  the  problem  of  approximating  an  unknown 
smooth  function,  given  a  set  of  data  consisting  of  measurements  on  the  function.  We 
are  especially  interested  in  the  case  where  the  number  of  data  is  fairly  large,  and  is 
subject  to  considerable  noise.  We  focus  on  a  method  for  solving  this  problem,  called 
the  method  of  penadized  least  squares,  which,  for  some  reason,  does  not  seem  to 
have  received  much  attention  in  the  Approximation  Theory  literature,  despite  the 
fact  that  it  often  seems  to  work  much  better  than  severed  other  more  commonly  used 
methods.  Our  aim  is  to  give  a  complete  and  self-contained  treatment  of  penalized 
least  squares,  including  a  discussion  of  how  to  choose  the  smoothing  parameter. 

Key  words:  Data  fitting,  smoothing,  least  squares,  penalized  least  squares 


1.  Introduction 

Suppose  /  is  an  unknown  function  of  one  or  more  variables,  and  that  we  are 
given  a  set  of  measurements 


z;  =  Kf  +  «i,  i  =  l,...,n,  (1) 

where  Aj , . . . ,  A„  are  linear  functionals  defined  on  some  linear  space  T  containing 
/,  and  where  ei,...,e„  Eire  measurement  errors.  Typically  the  A*  will  be  point- 
evaluation  of  the  function  or  one  of  its  derivatives,  although  other  functionals  are 
also  of  interest. 

Our  aim  is  to  approximate  /  using  a  finite  dimensional  subspace  S  of  T  spanned 
by  functions  B\ , . . . ,  Bk,  where  k  <  n.  For  each  c  :=  (cj , . . . ,  c*)T,  let 

k 

se  =  c,  B , , 

1=1 


(2) 


The  problem  is  to  devise  some  reasonable  means  of  choosing  the  coefficients.  Per¬ 
haps  the  best-known  approach  is  the  classical  method  of  least-squares  whereby  we 
minimize 

E,(e)  =  -  Zi}2  (3) 

71 1=1 

over  all  choices  of  c. 

While  straightforward  least-squares  fitting  is  often  appropriate,  for  some  data 
fitting  problems  it  produces  a  function  which  is  not  sufficiently  smooth  (for  an 
example,  see  [11]).  In  such  cases,  it  may  be  better  to  look  for  a  function  in  S  which 
minimizes  a  combination  of  the  goodness  of  fit  (3)  with  an  appropriate  measure  of 
smoothness  of  the  fitting  function. 

Many  reasonable  measures  of  smoothness  of  the  function  sc  can  be  expressed 
in  the  form 

J(c)  =  ctEc ,  (4) 

where  E  is  a  symmetric  nonnegative-definite  k  x  k  matrix.  For  typical  examples  in 
univariate  and  bivariate  data  fitting,  see  Section  5  and  Remark  5,  respectively. 

Suppose  now  that  A  >  0  is  given  and  that  E  ^  0.  Then  for  each  c  €  1R*,  let 

Pa(c)  =  XctEc  +  ^  ]T[A;sc  -  z,]2.  (5) 

«=1 

The  second  term  in  (5)  is  just  the  mean  square  error  in  approximating  the  data  by 
sc.  The  first  term  can  be  thought  of  as  a  penalty  term  which,  with  the  appropriate 
choice  of  E,  measures  the  smoothness  of  sc.  The  parameter  A  can  be  regarded  as 
a  smoothing  parameter. 

Definition  1.  The  penalized  least-squares  fit  of  the  function  f  based  on  data  z  is 
the  function  s\tX  :=  sc(>)  corresponding  to  the  choice  of  c( A)  which  minimizes  the 
functional  p\{c)  defined  in  (5). 

While  the  use  of  such  penalty  terms  seems  to  be  well  established  practice  in 
statistical  data  fitting  as  well  as  in  regularization  of  ill-posed  problems,  the  method 
seems  to  have  received  much  less  attention  in  Approximation  Theory  circles.  For 
more  on  the  history  of  penalized  least  squares  methods,  see  Remark  1  in  Section  6 
below. 

The  remainder  of  this  paper  is  organized  as  follows.  In  the  following  section  we 
establish  the  existence  and  uniqueness  of  solutions  to  the  penalized  least  squares 
problem,  and  also  discuss  the  special  cases  when  A  =  0  and  when  A  approaches 
oo.  In  Section  3  we  discuss  the  connection  between  the  choice  of  the  smoothing 
parameter  A  and  the  goodness  of  fit.  Section  4  is  devoted  to  an  automatic  procedure 
for  selecting  the  smoothing  parameter  A  called  generalized  cross  validation.  In 
Section  5  we  give  details  of  how  penalized  least  squares  can  be  applied  to  fit  data 
in  the  univariate  case  using  splines.  The  last  section  is  devoted  to  remarks. 


2.  Existence  and  uniqueness 

Throughout  the  remainder  of  this  paper  we  assume  that  the  linear  functionals 
Aj , . . . ,  A„  and  basis  functions  B\ , . . . ,  F*  are  such  that  for  some  1  <  v\  <  •  •  •  < 
Vk  <  n, 

det  (X„iBj)i j=1  ^  0.  (6) 

Under  this  basic  hypothesis,  we  can  now  establish  the  following  existence  and 
uniqueness  theorem  for  penalized  least-squares  fitting. 

Theorem  2.  For  any  A  >  0,  there  exists  a  unique  vector  c(A)  minimizing  the 
functional  p\(c)  in  (5).  In  particular,  c(A)  is  the  unique  solution  of  the  system 

(BtB  +  nXE)c(X)  =  BT  z,  (7) 

where  z  =  (zi, ...  ,z„)T  and  B  is  the  n  x  k  matrix  with  entries 

Bjj  —  A I B  j ,  ?  —  1, . . .  ,n;  j  —  1, ...,1c. 

Proofs  Setting  the  gradient  of  the  function  p\(c)  equal  to  0  leads  immediately 
to  the  system  of  equations  (7).  The  basic  hypothesis  (6)  assures  that  the  matrix 
G  :=  BTB  is  symmetric,  positive  definite,  and  nonsingular.  Since  E  is  assumed  to 
be  symmetric  nonnegative-definite,  we  conclude  that  for  every  A  >  0,  G  +  nXE  is 
also  symmetric  positive  definite.  ■ 

As  in  the  case  of  the  classical  smoothing  splines,  clearly  the  size  of  the  smooth¬ 
ing  parameter  A  has  a  major  impact  on  how  well  the  function  s\,z  fits  the  function 
/  which  generated  the  data.  In  Sections  3  and  4  below  we  shall  discuss  how  to 
select  A  effectively.  In  the  remainder  of  this  section  we  discuss  the  nature  of  the  fit 
in  the  extreme  cases  when  A  =  0  and  when  A  approaches  oo. 

When  A  =  0,  it  is  clear  that  the  functioned  p\{c)  is  just  the  mean  square  error 
when  fitting  the  data  using  functions  of  the  form  (2).  Hence  this  ceise  corresponds 
to  classiced  least-squares  fitting  using  the  space  S  =  span  {Hj}*,  and  the  system 
(7)  is  just  the  usued  set  of  normed  equations.  If  k  =  n,  then  the  least-squares  fit  is 
simply  the  function  in  S  which  interpolates  the  data. 

The  situation  as  A  approaches  oo  is  somewhat  more  complicated.  First  we 
prove  a  lemma  concerning  the  behavior  of  the  inverse  of  the  matrix  G  4-  nXE. 
where  G  =  BTB. 

Lemma  3.  For  all  A  >  0,  the  entries  of  (G  4-  nXE)~l  are  uniformly  bounded  by 
1/a,  where  a  is  the  smallest  eigenvalue  of  G. 

Proof:  Let  x  be  the  rth  column  of  ( G  4-  nXE)~l .  Then  (G  -I-  nXE)x  =  er,  where 
er  is  the  vector  in  Rk  with  all  zero  entries  except  for  the  rth .  It  follows  that 
xT(G  4-  nXE)x  =  xTer  =  xr.  Now  by  the  nonnegative  definiteness  of  E,  we  con¬ 
clude  that  xTGx  <  xT.  On  the  other  hand,  by  the  positive  definiteness  of  G,  the 
smallest  eigenvalue  a  of  G  satisfies 

»*o  yl  y 
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I 


It  follows  that  axTx  <  xTGx  <  xr,  and  thus  aij  <  xr,  for  all  j  =  1, . . . ,  k.  This 
inequality  asserts  in  particular  that  0  <  xr  <  1/a,  and  it  immediately  follows 
that  |xj|  <  1/a,  for  all  j  =  1  Since  r  was  an  arbitrary  integer  satisfying 

1  <  r  <  fc,  this  completes  the  proof.  M 

We  can  now  examine  the  behavior  as  A  — ►  oo  of  the  coefficient  vector  c( A) 
which  minimizes  the  functional  p\(c)  in  (5). 

Lemma  4.  There  exists  a  vector  c( oo)  in  Rk  such  that 

lim  e(A)  =  c(oo). 


Moreover,  Ec(oo)  =  0. 


Proof:  Clearly,  c(A)  =  (G  4-  nXE)~1BTz.  Since  the  entries  of  (G  +  n\E)~l  are 
rational  functionals  of  A  and  are  uniformly  bounded  for  all  A  >  0,  it  follows  that 
(G  +  n\E)~l  converges  to  some  matrix  L  as  A  — *  oo.  But  then  c(A)  approaches 
LBtz  as  A  — ►  oo.  Finally,  since 


BTz 
nX  ’ 


it  follows  that  Ec( oo)  =  0.  ■ 


Theorem  5.  The  function  sc(oo)  is  the  least-squares  fit  of  the  data  from  the  fol¬ 
lowing  subspace  of  S: 

$£  =  :  £c  =  °j.  (8) 

Proof:  Since 

Gc(A)  +  nXEc(X)  =  Btz, 

and  the  first  term  converges  as  A  — ►  oo,  we  conclude  that  nXEc(X)  converges  to 
u  :=  Btz  —  Gc(oo).  The  matrices  E  and  (E,  u)  have  the  seune  rank,  and  it  follows 
that  u  =  Ex  for  some  element  Ex  with  x  in  Rk.  Thus  we  have 


G  E 

'c(oo)' 

Btz 

E  0 

X 

0 

ajat* 


But  this  is  precisely  the  system  of  equations  which  uniquely  determines  the  least- 
squares  fit  of  the  data  from  the  subspace  Se-  H 


3.  Properties  of  the  fit 

In  this  section  we  discuss  the  behavior  of  the  fit  as  we  vary  the  smoothing 
parameter  A.  Given  measurements  as  in  (1),  let  s\yZ  be  the  penalized  least-squaxes 
fit  of  /  constructed  from  the  noisy  data  vector  z.  Then  the  mean-square  error  using 
this  fit  is  given  by 

r*(A)  =  -J>,(s>lS-/)]2-  0) 

n  .=i 

We  begin  by  considering  the  case  where  we  have  exact  measurements;  i.e., 
where  the  errors  ci,...,en  in  (1)  are  all  zero.  In  this  case  we  want  to  consider 
Tf(A),  where  f  -  ( \i /, . . . ,  A„/)T. 

Theorem  6.  The  function  7f(A)  is  monotone  increasing  for  A  >  0  with  Tf(0 )  =  0 
and  limA— oo  Tf(A)  =  0. 

Proof:  First  we  establish  the  monotonicity  of  Tf(A).  For  exact  measurements 
z  —  f ,  the  minimum  of  the  expression  p\  in  (5)  is  attained  at  Cf(A),  and  is  given  by 

Pa(ct(A))  =  Acf(A)T£cf(A)  +  Tf(A). 

Similarly,  for  all  A  >  A  >  0, 

Pi(cf(A))  =  Xcf(X)TEc((X)  +  7V(A). 

For  ease  of  notation,  we  now  define  a  =  Cf(A)r£7cf(A),  a  =  Cf(X)T Ec{(\),  /?  = 
Tf(A),  and  0  =  7f(A).  Then  the  minimality  of  Cf(A)  and  Cf(A),  respectively,  imply 
that 

Aa  +  0  <  Ad  +  0  and  Ad  4-  0  <  Aa  4-  0. 

It  follows  that 


Aa  -f  Ad  <  Ad  +  Aa  and  A(d  —  a)  <  A(a  —  a), 

and  thus  that  d  <  a.  Since  A(a  —  a)  <  (3  —  (3,  we  conclude  that  $  >  f3  which  asserts 
that  Tf(A)  is  monotone  increasing. 

To  establish  the  assertions  about  the  derivative  of  Tf(A),  we  need  an  explict 
formula  for  it.  To  this  end  we  introduce  the  n  x  n  influence  matrix 

A(X)  =  B(G  +  n\E)~lBT.  (10) 


By  the  properties  of  the  matrices  appearing  in  (10),  it  follows  immediately  that 
v4(A)  is  symmetric  positive  definite.  In  addition,  we  claim  that  .4(A)  provides  the 
connection  between  the  data  vector  z,  and  the  associated  values  of  the  penalized 
least  squares  fit;  i.e., 


An^A 


•  • 


l 


=  A(X)z. 


(ID 


Indeed,  (A, a*,,,  . . . ,  AnsA,z)T  =  Bc( A)  =  B(G  +  n\E)~lBTz  =  A(\)z. 
Now  it  is  clear  that 

r,(A)  =  i[-4(A)f-f|7'[A(A)f-f|. 

TX 

From  this  we  compute  that 

n[Xf(A)  -  Tf(0)3  =  fT[A(A)TA(A)  -  2A(A)  -  A(0)rA(0)  +  2A(0)]f. 
Using  (10),  we  arrrive  at  the  formula 


Tf(A)  =  Tf(0)  +  nA2Cf(A)T£,G~1F?cf(A).  (12) 

Forming  the  difference  quotient  of  Tf(A)  at  0  and  using  the  continuity  of  Cf(A), 
(as  shown  in  the  proof  of  Lemma  4),  it  immediately  follows  that  the  derivative  at 
A  =  0  is  zero  as  asserted.  The  assertion  about  the  derivative  as  A  — ►  oo  follows 
from  the  boundedness  of  Tf(A).  ■ 

It  is  now  clear  that  to  minimize  the  error  in  the  approximation  when  fitting 
exact  data,  we  should  choose  the  value  of  the  smoothing  parameter  A  to  be  zero. 
We  should  note,  however,  that  for  a  certain  class  of  functions  TE  which  we  are 
about  to  define,  2f(A)  is  constant,  and  hence  all  values  of  A  are  equally  as  good. 
The  class  is 

?E  =  {f  ■■  d(f,S)~d(f,SE)},  (13) 

where 

d(/,S)  =  inf  i  J>(S  - /)]*,  (14) 

j€$  n  f— : 

*=i 

and  d(f,SE )  is  defined  similarly.  Indeed,  in  this  case,  Tf(A)  has  the  same  value  for 
both  A  =  0  and  for  A  =  oo.  Thus,  since  Tf(A)  is  monotone  increasing,  it  must  be  a 
constant. 

In  practice  we  almost  always  have  noisy  measurements,  and  as  we  shall  see 
shortly,  in  this  case  it  will  be  more  advantageous  to  take  A  >  0.  To  analyze  the 
noisy  case,  we  need  to  make  some  basic  assumption  about  the  nature  of  the  noise. 
Throughout  the  remainder  of  this  section  we  suppose  that  the  errors  e\ , . . . ,  e„  in 
the  measurements  (1)  are  independent,  identically  distributed  random  variables 
with  mean  0  and  variance  a2 . 

It  is  easy  to  see  that 

nTt(A)  =  nTf(A)  +  2€rA(A)T[A(A)f  -  f ]  +  eTA(A)TA(A)e. 

Taking  the  expected  value  of  this  expression  gives 

er.(  A)  =  r,(A)  +  (is) 

n 

Before  analyzing  this  expression  further,  we  need  some  information  on  the 
trace  of  the  influence  matrix  A(A).  Let  0  <  <  «2  <•••<«*  be  the  eigenvalues 

of  G~1E,  and  suppose  that  {wi,...,v*}  are  the  corresponding  eigenvectors.  We 
suppose  that  kj  =  •  •  •  =  =  0  <  kj+i.  This  is  equivalent  to  assuming  that  the 

dimension  of  the  space  Se  defined  in  (8)  is  d. 


Lemma  7.  For  0  <  A  <  oo  the  matrix  A(A)  has  exactly  k  non-zero  eigenvalues 


_  1 

1  -f  n\Kj  ' 


For  A  =  oo,  the  eigenvalues  of  A(A)  consist  of  0  with  multiplicity  n  —  d  and  1  with 
multiplicity  d. 

Proof:  We  first  consider  the  case  where  0  <  A  <  oo.  Since  for  all  z,  the  vector 
A(A)z  lies  in  a  Jb-dimensional  subspace  of  lRn,  the  first  n  —  k  eigenvalues  of  A  are 
all  0.  Suppose  now  that  A  =  0,  and  fix  1  <  j  <  k.  Since  the  least  squares  fit  of  the 
function  Bj  is  Bj  itself,  it  follows  that  (Ai  Bj, . . . ,  A nBj)T  is  an  eigenvector  of  A(0) 
corresponding  to  eigenvalue  1.  This  proves  that  the  k  nonzero  eigenvalues  of  A(0) 
are  all  1. 

We  now  treat  the  case  where  0  <  A  <  00.  Suppose  v  6  IR*  is  an  eigenvector 
of  G~1E  corresponding  to  k  >  0;  i.e.,  G-1Ev  =  kv.  Let  p  =  and  suppose 

u  6  IR”  is  such  that  G~1BTu  =  v.  Let  w  =  Au.  Since  B  is  of  rank  k, 

Au  =  0  iff  Btu  —  0.  (16) 


The  fact  that  v  ^  0  coupled  with  (16)  assures  that  ui  /0.  Now  it  is  easy  to  see 
that  the  following  equalities  are  equivalent: 


kv  =  G  1  Ev. 


Gv  =  pGv  +  n\pEv 
Gv  =  p(G  +  n\E)v 
(G  +  n\E)~1BTu  =  pG~1BTu 
G(G  +  nXE)~1BTu  =  pBTu 
Bt(Au  —  pu)  =  0 
A(Au  —  pu)  =  0 
Aw  =  pw. 

This  establishes  that  p  is  an  eigenvector  of  A.  To  complete  the  proof  we  have  to 
consider  the  possibility  that  k  is  an  eigenvalue  of  G~lE  of  multiplicity  m.  Suppose 
i?i , . . . ,  vm  are  linear  independent  eigenvectors  of  G~1E  associated  with  k.  For 
each  i  —  l,...,m,  let  u,  €  IR"  be  such  that  G~*BTUi  =  Vi ,  and  let  u>,-  =  Au,. 
Now  we  claim  that  the  W{  are  linearly  independent  eigenvectors  of  A.  Indeed,  if 

a\W\  H - 1-  amwm  =  0,  then  o-i Aui  H - 1-  amAum  =  0,  and  by  (16),  it  follows 

that  a\G~l BT uj  H - \-amG~l BTum  =  aiVi  H - |-amum  =  0.  But  by  the  linear 

independence  of  the  v's,  this  implies  oj  =  •  •  •  =  am  =  0,  and  we  have  established 
the  asserted  linear  independence  of  the  toj’s. 


It  remains  to  deal  with  the  case  A  =  oc.  In  this  case,  we  are  performing  least 
squares  in  Se,  and  hence  s\  lies  in  a  d  dimensional  subspace  of  IR”.  It  follows  that 
n  —  d  of  the  eigenvalues  of  A(oo)  must  be  zero.  On  the  other  hand,  since  each  of 
the  d  basis  functions  of  Se  is  approximated  exactly,  just  as  in  the  case  A  =  0  we 
see  that  1  is  an  eigenvalue  of  A(oo)  of  multiplicity  d.  ■ 

For  later  use,  we  note  that  since  the  trace  of  a  matrix  is  equal  to  the  sum  of 
its  eigenvalues,  for  0  <  A  <  oo, 


,(A)  :=  trace  = 


(17) 


and,  since  the  eigenvalues  of  A2(X)  are  the  squares  of  the  eigenvalues  of  .4(A), 


<2(A)  :=  trace  A!(A)  =  g^_L_)  . 


(18) 


In  particular,  #(0)  =  *2(0)  =  k.  We  also  note  that  #(oo)  =  <2(00)  =  d. 

Theorem  8.  The  function  <p(X)  :=  £TZ( A)  has  the  value  £TZ(0)  =  Xf(0)  +  kcr2/n, 
and  asymptotically  approaches  the  value  Tf(oo)  4-  do2 /n  as  A  — ►  00.  Its  derivative 
at  X  =  0  is  negative.  Moreover,  for  all  X  >  0, 

£T,(X)  -  £TZ{ 0)  >  ^MA)  -  #2(0)],  (19) 

Tt 

where  #2(A)  is  defined  in  (18). 

Proof:  Putting  #2(0)  =  k  and  #2(00)  =  d  in  (15),  we  immediately  obtain  the 
asserted  values  of  <j>  at  A  =  0  and  A  =  00.  By  (15), 


Now  since 


*(A)  =  Tf(A)  + 


#2(A)  =  -2£ 


<r2i2(X) 


n 


nKi 


(20) 


"  (1  +  nA«;,)3 

and  7f(0)  =  0  by  Theorem  6,  we  immediately  deduce  that  </>(0)  <  0  as  asserted. 
Finally,  to  prove  (19),  we  note  that  by  (12)  and  (15), 

£TZ( A)  -  £7U0)  =  nX2cf(X)TEG-iEc{(X)  + 


Since  the  first  term  on  the  right  is  nonnegative,  the  result  follows. 


It  is  clear  from  (20)  that  in  the  case  where  /  €  Fe,  the  function  £TZ{X )  is 
monotone  decreasing  since,  as  observed  earlier,  Tf(A)  is  constant  in  this  case  while 
#2(A)  <  0. 


4.  Choosing  the  smoothing  parameter  A 

In  the  previous  section,  we  have  examined  the  connection  between  the  choice 
of  the  smoothing  parameter  and  the  size  of  the  true  error  or  expected  value  of  the 
true  error.  We  can  summarize  our  observations  as  follows: 

Case  1:  (The  noise  is  zero)  In  this  case,  in  order  to  minimize  the  true  error  of  fit, 
we  should  take  A  =  0;  i.e.,  we  should  perform  least  squares  using  the  space  S. 

Case  2:  (Noisy  data)  In  this  case  it  is  reasonable  to  choose  A  to  minimize  the 
expected  value  £TZ{ A)  of  the  true  error.  There  are  two  subcases.  If  £TZ( A)  is 
monotone  decreasing  on  [0,  oo),  then  we  should  take  A  =  oo.  On  the  other  hand, 
if  £TZ( A)  is  not  monotone  decreasing  on  [0,  oo),  then  since  by  Theorem  8,  £TZ( A) 
has  a  negative  derivative  at  A  =  0,  there  is  at  least  one  value  of  A  in  (0,  oo)  which 
minimises  £TX( A).  Let  A^  be  the  smallest  choice  of  A  which  provides  a  minimum. 

In  the  remainder  of  this  section  we  restrict  our  attention  to  the  case  of  noisy 
data.  Our  immediate  aim  is  to  discuss  an  automatic  method,  called  generalized 
cross  validation,  for  computing  a  reasonable  estimate  for  A  J.  Given  a  data  vector  z 
as  in  (1),  let  s\tZ  be  the  associated  penalized  least-squares  fit,  and  let  A(  A)  be  the 
influence  matrix  defined  in  (10).  Then  we  define  the  associated  generalized  cross 
validation  function  as 

V<A>  -  (21) 


where 


E,W=-T(XiSX,z-zi)2, 

n  * — ' 


and  t( A)  is  the  trace  of  .4(A)  as  given  in  (17). 

Lemma  9.  The  function  Ez( A)  is  monotone  increasing  for  A  >  0  with  Ez( 0)  =  0 
and  lim*  — >oo  Ez(  A)  =  0. 

Proof:  The  proof  of  this  lemma  is  virtually  the  same  as  the  proof  of  Theorem  6, 
and  we  omit  the  details.  ■ 

Lemma  9  does  not  assert  that  Ez(  A)  is  strictly  monotone  increasing.  For  some 
z  it  can  happen  that  Ez( A)  is  constant  for  all  0  <  A  <  oo.  We  now  show  that  V{\) 
has  properties  very  similar  to  those  of  £TZ( A). 

Theorem  10.  The  function  F(A)  has  value  Ez( 0)/[l  —  k/n ]2  at  0  and  asymptot¬ 
ically  approaches  the  value  Ez( oo)/[l  —  d/n]2  as  A  — ►  oo.  Moreover,  F(A)  has  a 
negative  derivative  at  A  =  0. 

Proof:  The  first  assertions  follow  immediately  from  the  properties  of  t( A).  To 
prove  the  last  assertion,  we  note  that 


V(X)  = 


n2(n  -  t(X))Ez(X)  -1-  2n2Ez(\)t(X) 
[n  —  1(A)]3 


The  assertions  now  follow  from  the  properties  of  Ez( A)  and  the  fact  that 


<(A)  =  -£ 


TlKi 


(1  +  nA/c,)2 


Theorem  10  imphes  that  either  V(A)  is  monotone  decreasing  on  [0,  oo)  in  which 
case  we  can  think  of  its  minimum  as  occuring  at  Aj(  :=  oo,  or  there  is  at  least  one 
value  of  A  in  (0,oo)  where  Ez  has  an  absolute  minimum,  in  which  case  we  define 
A^  to  be  the  smallest  choice  of  A  which  works. 

The  process  of  computing  the  quantity  A^  as  an  approximation  to  A^  is  called 
the  method  of  generalized  cross  validation.  To  implement  it  numerically,  we  have 
to  find  the  minimum  of  the  validation  function  V(A).  The  standard  approach 
to  doing  this  is  to  compute  V(A)  at  several  choices  of  A,  and  then  use  a  search 
procedure.  For  each  choice  of  A,  the  bulk  of  the  computational  effort  typically  goes 
into  finding  f(A)  =  trace  A(A).  In  those  cases  where  the  matrix  B  has  a  band 
structure  (as  happens  for  example  when  using  univariate  splines  -  see  Section  5 
below),  the  trace  can  De  computed  efficiently  using  the  LU  decomposition  of  the 
matrix  ( G  +  nXE),  see  [13,21].  In  the  general  case  it  may  be  more  efficient  to  resort 
to  finding  the  singular  value  decomposition  of  ( G  +  nXE),  which  then  leads  to  an 
explicit  expression  for  t( A),  see  [2]. 

From  a  theoretical  standpoint,  it  is  of  interest  to  consider  the  expected  value 
of  the  validation  function  V .  Since 


£EZ(  A)  =  Tf(A)  +  — t3(A), 
n 


where 

*3(A)  =  trace  (A  -  I)2, 


we  obtain 


£V(X)  = 


Tf(A)  + 
[i  - 


The  following  theorem,  whose  proof  is  straightforward,  shows  that  £V(X)  has  prop¬ 
erties  very  similar  to  those  of  both  £TZ{ A)  and  F(A). 


Theorem  11.  The  function  £V(X)  has  the  value  a *  0,  and  sisymptotically 

approaches  the  value  as  A  — *  oo.  Its  derivative  at  X  =  0  is  negative. 


Theorem  11  shows  that  either  the  function  £V(X)  is  monotone  decreasing  for 
all  A,  or  there  is  some  value  of  0  <  A  <  oo  where  it  has  a  minimum.  We  denote 
the  first  such  point  by  X%v .  In  practice,  we  cannot  compute  this  value  because  we 
have  no  way  of  computing  either  Tf(A)  or  <r.  (Indeed,  if  we  could  compute  these 
quantities,  we  could  find  the  minimum  of  £TZ( A)  directly).  We  have  the  following 
interesting  theorem  concerning  A„  v. 


•  • 


i 


<1 


Theorem  12.  The  quantity  \„v  is  an  asymptotically  optimal  estimate  of  the  best 
smoothing  parameter  A  ^  in  the  sense  that 


ST,(\ 

-  ST,(\T) 


1 


as  n  —y  oo. 

Proof:  As  in  [23],  it  is  easy  to  show  that  for  A  >  0, 


(23) 


,  -  ST.(\*r)  1  +  A„(Al) 

-  £T,(AJ)  -1-A,(»f)’ 


(24) 


where 


A„(A) 


f(A)2  -  2nt(\)  +  n<(A)2/<2(A) 
[n-t(A)P 


and  where  <(A)  and  £2(A)  are  defined  in  (17)  and  (18),  respectively.  Now  since 
d  <  t( A)  <  k  and  d  <  t2(A)  <  k,  it  follows  that  the  quotient  in  (24)  converges  to 
zero  at  a  rate  O(^)  as  n  — ♦  oo.  ■ 


For  smoothing  splines,  the  only  known  way  of  producing  sequences  An  which 
are  asymptotically  optimal  is  via  generalized  cross  validation.  The  following  result 
shows,  however,  that  for  penalized  least  squares  fitting,  the  situation  is  different. 
Here  there  is  a  range  of  values  of  A„  which  are  asymptotically  optimal. 


Theorem  13.  Suppose  that  for  each  n  >  1,  the  quantity  A 
£TZ( A„)  <  £TZ(0).  Then 


lim 

n— mx> 


€TZ(  An) 

£TZ{ \l) 


=  1. 


n 


>  0  is  such  that 
(25) 


Proof:  By  (19)  and  the  fact  that  <2(0)  —  t2(A)  <  k  —  d,  we  have 


.  <  STZ{ An)  STZ{ 0) 

-  £TZ( \T)  ~  STZ( 0)  -  (Jk  -  d)^/n 


Clearly  this  expression  converges  to  1  at  a  rate  of  O(^).  ■ 


Theorem  13  asserts  in  particular  that  An  =  0  provides  an  asymptotically  op¬ 
timal  sequence  of  smoothing  parameters.  In  fact,  since  £TZ( A)  has  a  negative 
derivative  at  A  =  0,  any  sequence  of  ’’sufficiently  small”  numbers  A„  would  work. 

The  fact  that  £T*(0)  —  (k  —  d)o2/n  <  £TZ{ A)  <  £TZ( 0)  for  all  sufficiently 
small  A  suggests  that  for  fairly  large  values  of  n,  there  is  little  to  gain  in  terms  of 
goodness  of  fit  by  striving  for  the  optimal  value  of  A;  it  is  enough  to  make  sure 
that  A  is  sufficiently  small.  On  the  other  hand,  for  noisy  data,  in  practice  it  is  often 
necessary  to  take  A  >  0  in  order  to  get  a  smooth  fit.  Generalized  cross  validation 
can  be  a  useful  automatic  procedure  for  selecting  a  good  value  of  A. 


L 

I 

L 

5.  Penalized  Least  Squares  Fitting  of  Curves  Using  Splines 

|  In  this  section  we  show  how  the  penalized  least  squares  method  can  be  applied 

to  fit  a  spline  to  the  data 

i  A*7  =  /(*«)  +  £.,  i  = 

I  where  /  €  C[a,  6],  and 

a  =  ti  <•••</„  =  b. 

j  Let  m,k  >  0  be  prescribed  integers,  and  suppose  that 

Bi(x)  =  A?m(x),  (26) 

^  are  the  normalized  B-splines  of  order  m  associated  with  an  extended  knot  vector 

yi  <  •••  <  1/2 m+fc,  with  yi  <  yi+2m  for  all  i  (cf.  [20]).  We  may  assume  that  the 
I  knots  have  been  chosen  in  such  a  way  that  j/2m  <  a  and  b  <  j/t+i,  and  so  that  (6) 

*  is  satisfied.  By  the  Schoenberg-Whitney  Theorem  (cf.  Theorem  4.64  of  [20]),  (6) 

(will  be  satisfied  provided  we  choose  the  knots  so  that  there  exist  1  <  1/2  <  •  •  •  < 
i/k-i  <  n  so  that  y,  <  <  y,+TO  for  all  i  =  2, . . . ,  k  -  1. 

When  using  splines,  a  natural  measure  for  the  smoothness  of  a  fit  s  would  be 

[Dmg(i))2dt.  (27) 

In  this  case,  if  s  is  a  spline  of  the  form 


k 

3(x)  -  "^CiBiix), 
i=i 

then  J(g )  =  cTEc  with 

Eij  =  /  Dm Bi(t)Dm Bj(t) dt,  i,j  =  l,...,Ar.  (28) 

Ja 

We  now  discuss  the  problem  of  computing  the  quantities  appearing  in  (28). 
First  we  show  how  to  represent  the  derivative  of  a  B-spline  in  terms  of  lower  order 
ones. 

Lemma  14.  Let  Nf(x)  be  the  normalized  B-splines  of  order  i  associated  with  the 
knot  sequence  &<■••<  y with  y,  <  yi+(  (cf.  [20]).  Define  hi<y  =  yi+v  -  y,. 
Then  for  any  q  >  0, 

d'nH*)  =  (29) 

u=0 


’  l 


f 


where  the  Q ’s  sire  unnormalized  B-splines  on  the  same  knots,  and  where  the  a \  u ’ s 
can  be  computed  recursively  as  follows: 


1.  Set 


2.  Do  v  =  2  to  q 


Av  = 


(l,  v  =  2 
-1,  v  =  Z 
10,  i/  =  l,4,... 


q  +  2. 


Do  p  =  v  +  2  step  —1  to  2 


Am  =  (1 


-4u-i 


-) 


3.  Do  i/  =  0  to  q 
Oitu  =  foC  *  j4|,+2i 

where  fac  =  (£  —  l)(f  —  2 )•■•(£  —  q).  If  any  of  the  h's  is  zero  in  step  2,  the 
corresponding  term  should  be  omitted. 

Proof:  This  result  follows  easily  from  the  basic  formula  (cf.  Theorem  4.16  in  [20]) 


DQ‘i(x)  =  (e-  1) 

and  the  fact  that  Nf(x)  =  hijQ^x).  I 
Theorem  15.  Foralli,j  we  have 


(rw-c  (*) 

hi,e 


t- 1/ 


rb  mm  .6 

Eij=  /  DmQ?m(x)DmQ^(I)(ix  =  EE 

Jo  *=0u  =  0  •/a 


Proof:  The  result  follows  easily  from  Lemma  14.  ■ 

Theorem  15  can  easily  be  converted  into  an  algorithm  for  computing  the  entries 
of  the  matrix  E.  Let 


Q?{x)Q™{x)dx  1  .  (30) 

'  i,i= 1 

This  is  a  2m  —  1  banded  symmetric  matrix  which  can  be  computed  accurately  and 
efficiently  using  Gauss  quadrature  (see  [4,20]).  Then 

E*,j  —  aJ P{hj)aji 

where  P(i,  j )  is  the  (m  +  l)x(m  +  l)  minor  of  P  obtained  by  taking  rows  i, . . . ,  i+m 
and  columns  j,. . .  ,j  +  m,  and  where  a<  =  [a™0, . . . , a™m]T  for  all  i. 

Penalized  least  squares  using  cubic  splines  with  uniform  knots  has  been  used 
in  [11]  to  fit  some  medical  data,  and  in  [15]  to  fit  some  mechanical  data. 


6.  Remarks 


Remark  1 

The  idea  of  minimizing  a  combination  of  goodness  of  fit  and  some  measure 
of  smoothness  is  well-known  in  the  approximation  literature.  Such  expressions 
arise,  for  example,  in  the  definition  of  smoothing  splines  (see  [5,13-14,16-25,31]) 
and  of  thin  plate  splines  (see  [7-10,25,28-30]).  In  these  methods  the  approximating 
function  turns  out  to  be  a  linear  combination  of  n  basis  functions  (see  Remark 
3  below).  The  idea  of  working  with  a  smaller  number  k  of  basis  functions  was 
explicitly  mentioned  in  Wahba  [26-28],  but  seems  to  have  been  carried  out  explicitly 
in  only  a  few  papers.  In  curve  fitting  case,  cubic  splines  with  equally  spaced  knots 
were  used  in  [11]  to  fit  some  medical  data,  and  in  [15]  to  fit  some  mechanical  data. 
Penalized  least  squares  methods  have  also  been  used  to  fit  surfaces  to  scattered 
data  using  tensor- product  splines  [6]  and  finite  elements  defined  on  rectangles  [1]. 

Remark  2 

The  method  of  penalized  least  squares  as  described  here  is  closely  related  to  the 
method  of  ridge  regression  as  studied  by  statisticians  (see,  for  example,  [12]).  The 
ridge  regression  problem  is  to  minimize  the  expression  p\(c)  in  (5),  with  E  replaced 
by  the  identity  matrix.  It  has  been  shown  in  [2]  that  the  general  penalized  least 
squares  problem  can  be  reduced  to  the  ridge  regression  problem  by  appropriate 
matrix  manipulations. 

Remark  3 

Suppose  a  =  t\  <  ■  •  •  <  t„  =  b  as  in  Section  5.  Then  if  we  minimize 

a  /  in*)]’* +  i£i/(«  -*]’ 

Ja  »=i 

over  all  functions  with  square  integrable  m-th  derivative,  we  get  the  classical  natural 
smoothing  spline  (see  [5,13-14,16-25,31]).  The  theory  of  smoothing  splines  can 
be  regarded  as  a  special  case  of  the  penalized  least  squares  method  treated  here. 
Indeed,  if  we  set  k  —  n  and  take  the  natural  B-splines  (cf.  [14.20])  as  basis  functions, 
then  the  solution  of  the  penalized  least  squares  problem  is  precisely  the  smoothing 
spline.  Almost  all  of  what  we  have  done  here  is  valid  for  the  case  where  k  =  n  with 
the  exception  of  the  assertions  about  asymptotic  optimality  (see  Remark  4). 

Remark  4 

The  asymptotic  assertion  of  Theorems  12  and  13  have  been  proved  here  only 
for  the  case  where  k  is  fixed.  However,  it  is  not  hard  to  see  that  the  results  are 
also  valid  for  a  sequence  k(n)  with  the  property  that  k(n)/n  — »  0  as  n  — *•  oo.  The 
situation  when  k(n)/n  does  not  go  to  zero  as  n  — >  oo  is  more  delicate.  For  example, 
to  prove  the  analog  of  Theorem  12  for  natural  smoothing  splines  (see  [22])  or  for 
complete  smoothing  splines  (cf.  [21]),  it  was  necessary  to  give  precise  estimates  on 
the  eigenvalues  of  E. 


Remark  5 

In  Section  5  we  showed  how  to  use  univariate  splines  to  fit  noisy  data  in  one 
dimension  by  penalized  least  squares.  Clearly,  the  method  can  also  be  used  to  fit 
surfaces  to  noisy  data.  For  example,  suppose  /  is  defined  on  some  subset  ft  C  IR2. 
Then  we  may  measure  the  smoothness  of  possible  fits  g  of  /  by 

J{s)  =  (ff> 9)m,  (31) 

where,  in  general, 

:=  /  DvzD*<t>{x,y)DvzD»xp(x,y)dxdy.  (32) 

v+H=m  " 

In  this  case,  if  s  is  as  in  (2),  then  J(s )  =  cTEc  with 

Eij  =  (5,,  Bj )m  • 

Penalized  least  squares  of  surfaces  can  be  performed  using  a  variety  of  bases  includ¬ 
ing  tensor  product  splines  (cf.  [6]),  radial  basis  functions  (cf.  [26]),  finite  elements 
(cf.  [1]),  or  multivariate  splines  on  triangulations. 

The  energy  expression  (31)  arises  in  the  definition  of  thin  plate  splines  (see  [7- 
10,25,28-30]  and  references  therein).  Thin  plate  splines  can  be  regarded  as  arising 
from  penalized  least  squares  using  k  =  n  and  appropriate  basis  functions. 

Remark  6 

Generalized  cross  validation  methods  have  been  heavily  studied  in  the  case  of 
spline  smoothing  (cf.  [5,13-14,16-25,31])  as  well  as  for  ridge  regression  [12]  and  for 
regularization  of  ill- posed  problems  [27].  In  connection  with  general  penalized  least 
squares  problems,  generalized  cross  validation  is  treated  in  [2].  It  was  also  used  in 
[15]  in  connection  with  a  cubic  spline  method.  The  basic  statistical  hypothesis  that 
the  errors  are  independent  identically  distributed  random  variables  is  standard 
in  discussing  such  methods.  We  should  note  that  we  are  not  assuming  that  the 
distributions  are  Gaussian,  only  that  they  are  all  the  same.  However,  even  this 
hypothesis  may  well  not  be  satisfied  in  some  practical  situations. 

Remark  7 

Error  bounds  for  spline  smoothing  can  be  found  in  [16,19]  and  for  thin  plate 
splines  in  [25].  Error  bounds  for  penalized  least  squares  fitting  of  surfaces  using 
finite  elements  can  be  found  in  [1].  We  intend  to  treat  error  bounds  for  penalized 
least  squares  in  a  separate  paper. 


Remark  8 

In  some  practical  problems,  in  forming  the  error  expression  (3),  it  may  be 
desirable  to  weigh  the  t-th  measurement  with  a  weight  factor  to,-  >  0  for  t  =  1, . . . ,  n. 
In  this  case,  the  results  of  the  paper  remain  valid  with  only  minor  adjustments. 
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Abstract  The  general  linearly  constrained  best  approximation  problem  with  arbi¬ 
trary  norm  is  cast  as  a  semiinfinite  linear  programming  problem,  using  a  device  of 
A.  Haar  (1918)  .  The  resulting  optimization  problem  -  that  of  maximizing  a  linear 
form  over  a  convex  set  described  by  the  intersection  of  its  supporting  halfspaces  - 
can  be  solved  numerically  by  a  2-phase  hybrid  “interior  point  /  cutting  plane”  algo¬ 
rithm.  A  brief  outline  of  both  phases  of  the  algorithm  is  given.  Phase  1  is  concerned 
with  an  “activity”  analysis  of  the  constraints  posed  on  the  desired  fit,  and  with  the 
construction  of  an  initial  feasible  solution.  Phase  2  produces  a  sequence  of  feasible 
(interior)  and  infeasible  (exterior)  solutions,  both  sequences  converging  to  the  opti¬ 
mal  constrained  solution.  The  method  is  competitive  in  non-Lj  settings,  especially 
when  the  norm  is  not  smooth  and/or  the  constraints  posed  are  particularly  active. 
Key  words:  Constrained  approximations,  Semiinfinite  programming,  Interior  point 
algorithm. 


1.  Introduction 
1.1  Rationale 

Researchers  in  the  empirical  sciences  are  faced  with  the  general  problem  of  “fitting” 
data  in  a  pleasing  ,  rational,  and  efficient  manner.  One  means  of  attack  is  to  cast  the 
data  fitting  problem  as  a  constrained  best  approximation  problem  in  a  normed  linear 
space.  The  constraints  should  express  various  shape  qualities  and  interpolation 
conditions  that  the  data  fit  must  possess  to  be  “pleasing”  or  useful.  The  choice 
of  norm  is  dictated  by  both  the  form  of  the  hypothesized  or  suspected  errors  on 
the  data  and  by  the  application  intended  for  the  fit  after  it  has  been  produced. 
The  choice  of  approximating  space  is  made  on  the  basis  of  one  or  several  of  the 


following  criteria:  computational  ease,  model  predictions,  degree  of  approximation, 
and  manipulative  ease  in  later  applications. 

Although  algorithms  exist  for  solving  constrained  best  approximation  problems 
for  certain  specific  combinations  of  constraints,  norm,  and  approximating  family, 
no  general  all  purpose  algorithm  has  been  available.  This  is  the  motivation  for  the 
present  work;  framing  the  constrained  approximation  problem  as  a  semiinfinite  con¬ 
vex  programming  problem  posessing  special  structure,  and  developing  a  numerical 
algorithm  that  could  then  be  applied  in  any  situation  that  may  obtain. 

1.2  The  linearly  constrained  best  approximation  problem 

In  this  section  the  data  fitting  problem  is  cast  in  a  mathematical  framework,  flexible 
enough  to  handle  most  practical  situations.  Let  {/i,/2, . .  be  n  linearly  inde¬ 
pendent,  real  valued  functions  defined  and  continuous  on  the  compact  set  I  C  Rk. 
The  linear  space  V  :=  lin{/i,  /2, . . .,  /„}  will  be  referred  to  as  the  approximating 
subspace,  and  the  set  {/i,  ft, . . .,  /„}  as  the  approximating  family.  Let  /  denote 
real  data,  defined  and  continuous  on  the  set  I;  i.e.,  /  e  C[I].  If  one  wishes  to  fit  the 
data  f  using  elements  from  V,  one  needs  a  criterion  for  measuring  the  goodness  of 
the  fit;  this  measure  is  usually  a  norm  or  seminorm  defined  on  the  linear  space  W 
of  functions  gotten  by  adjoining  /  to  V  ;  and  is  denoted  by  ||A||,  where  h£W.  The 
problem  of  finding  gba  €  V  such  that: 

117-^11  =  inf  ||/- *|| 

is  the  classical  problem  of  best  approximation  from  a  linear  subspace.  The  element 
gin  is  called  the  best  “||  ||  -  approximation”  to  /  from  V.  Using  standard  compactness 
and  continuity  arguments,  one  can  easily  prove  the  existence  of  such  an  element 
gba  €  V  for  each  /  6  C[I]. 

However,  one  often  desires  that  the  best  fit  to  the  data  also  possess  other  shape  or 
structural  qualities.  In  a  large  majority  of  cases,  such  constraints  can  be  formulated 
in  terms  of  an  indexed  collection  of  linear  constraints  of  the  form: 

a(p)  <  L(p\  f,g)  <  b(p),  ip  e  J 

where: 

a(p)  IT, 
b(p) 

with  Re  the  extended  reals;  L(p\  /,  g)  is  a  continuous  linear  functional  acting  on  /  or 
g  G  V  or  both,  indexed  by  and  continuous  in  p  €  J;  and  J  is  a  compact  topological 
space,  called  the  index  set,  often  identical  with  I  or  a  subset  of  I.  The  function  a 
(reap,  b  )  is  required  to  be  u.s.c.  (resp.  l.s.c.)  on  J,  continuous  on  J  —  o_,{-oo} 
(reap.  J  -  6_l{oo)  ),  and  to  avoid  vacuous  problems,  o(p)  <  b(p),  'ip  €  J. 

In  the  case  where  I  C  Rl  ,  one  has  as  common  examples  of  such  linear  constraints 
(assuming  g  6  V  and  /  are  sufficiently  smooth): 
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1.  Interpolation  Conditions:  the  approximant  g  is  to  exactly  agree  with 
the  data  /  at  the  specified  points  p  €  J  C  I.  Then  take  a(p)  =  b(p )  =  0, 

and 

£(p;  f,g)  •■=  f{p)  -  9(p),  Vp  s  J. 

2.  Non- negativity  Constraints:  the  approximant  g  is  to  remain  non-negative 
over  some  subset  J  C  I.  Set  a(p)  =  0,  6(p)  =  oo,  and 

Mp;/,9)'-=  g(p),  Vpe  J- 

3.  Specified  Moments:  the  approximant  g  is  to  have  its  p^  moments,  p  6 
J  C  Z,  specified  as  either  the  same  as  that  of  the  data,  /,  or  as  some  value 
mp  .  In  the  first  instance,  take  a(p)  =  b(p)  =  0  for  p  G  J,  and 

i(Pi/,5):=  f  xp(f(x)~  g(x))dx. 

•'I 

In  the  second  instance,  take  a(p)  =  6(p)  =  mp  for  p  €  J,  and 

L(p;f,9)  :=  Jixpg(x)dx. 

Notice  that  in  examples  1  and  2,  it  happened  that  J  C  I,  whereas  example 
3  shows  that  J  may  just  as  well  be  an  arbitrary  index  set.  One  is  free  to  impose 
several  of  the  above  conditions  simultaneously  in  any  approximation  problem,  and 
then  the  index  set  J  might  contain  several  “copies”  of  I  or  subsets  of  I. 

1.3  Haar’s  transcription 

To  transcribe  the  constrained  best  approximation  problem: 

mi%ev  11/  -  ff|| 

Vc:> 

,  o(p)  <  L(p\f,g)  <  Hp ),  Vp  e  J 

into  a  semiinfinite  linear  program,  one  introduces  homogeneous  coordinates  x  := 
(ii,. .  .,in,z„+i),  xn+i  >  0,  as  follows.  For  each  g  €  V,  g  =  £"=i  a,/,,  and 
f~9  —  /— £"=  i  <*»/«  ,  so  e  :=  f-g  €  W.  In  W,  consider  now  all  linear  combinations 
of  the  form: 

n 

W  =  52  Xifi  +  xn+lf,  xn+i  >  0, 
i=i 

which  we  write  as: 

«>  =  *■/,  e„+i  •  x  >  0 
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where: 

• 

i  :=  (Z), . .  .,zn, xn+j), 

/  -=  (/l»  •  •  •  t  fny  /)> 

«n+i  :=  (0, . . .,0, 1), 

• 

and  the  dot  product  is  the  standard  scalar  product  on  Rn+l  .  Homogeneous  is  a 
suitable  adjective  for  the  z  coordinates,  as  each  ray  :=  {x  |  x  =  tu,  t  >  0}  C  H, 
where  H  :=  {x  |  en+i  ■  x  >  0}  ,  corresponds  to  precisely  one  error  function  e  := 

/  -  9,  g  =  E?=ia*7*  *  via  the  wel1  defined  map:  a <  =  -z,/xn+i  =  -u,/un+1,  i  = 

Now,  defining  (3,{p)  :=  L(p;0,/,)  for  i  =  and  /3„+i(p)  :=  L(p;f, 0)  ,  the 

collection  of  constraints  in  Vc  becomes,  in  homogeneous  coordinates: 

n 

a(p)xn+x  <  -  £  A(p)*,  +  Pn+i(p)xn+t  <  b(p)xn+u  Vp  6  j. 

1=1 

• 

• 

Defining  the  vectors: 

n+(p)  :=  (Pl(p), .  ■  • ,  0n(p),  Pn+l(p)  —  fi(p)), 
for  all  p  6  J  -  a~l  {-oo}, 
n-(p)  :=  (~Pi{p),  •  •  •  i  ~/3n(p),  -/3„+i(p)  +  6(p)), 
for  all  p  G  J  -  6“l{oo}, 

• 

with  n+(p)  :=  0„+i  on  a-1  {— oo}  and  n_(p)  :=  0n+i  on  6-1{oo},  the  homogeneous 
variable  transcription  of  the  constraints  is  the  closed  convex  cone: 

• 

C  :=  {z  G  fln+1  |  n+(p)  •  x  <  0,  n_(p)  •  z  <  0,  Vp  G  J}. 

That  is  to  say,  any  vector  z  G  C  C\  H  corresponds  under  the  above  rule  of  corre¬ 
spondence  to  a  function  g  G  V  satisfying  the  totality  of  constraints  imposed  on  Vc. 

Moreover,  any  g  G  V  not  satisfying  some  constraint  has  homogeneous  coordinates 
not  in  C  D  H. 

Lastly,  define  the  closed  convex  body  (called  by  Haar  the  “Eichkorper”): 

• 

K  :=  {x  G  Rn+*  |  ||x  •  /j|  <  1}, 

• 

and  consider  the  semiinfinite  linear  program: 

v  j  max  e„+i  •  z 
h'\  £€Kr>C 

• 

With  the  above  notation  and  definitions,  one  has  the  simple  result: 

Result  1.  Problem  Vc  and  Vh  are  equivalent  in  the  following  sense: 

1.  Ve  is  inconsistent  if  and  only  if  Vh  has  optimal  solution  Xopt  =  0n+j. 

•  • 

'■  I  . 

i  " 

> 

2.  Vc  is  consistent  and  has  ||/  -  gopt||  =  0  if  and  only  if  Vh  has  unbounded 
feasible  solutions  and  unbounded  objective  values;  that  is,  there  exists  a  ray 
rs  :=  {x  =  tu,  t  >  0,  en+i  u  >  0}  C  KnC.  In  this  case,  any  <7opt  corresponds 
to  some  such  u  under  the  map  g  =  £"=i  ai Si  with  a,  :=  — u,/un+i  and  vice 
versa:  any  such  ray  rs  yields  an  optimal  solution  g^t  by  the  same  map. 

3.  Vc  is  consistent  and  has  \\f  -  gopt\\  =  E  >  0  if  and  only  if  Vh  has  a  finite 

optimal  solution  xopt  =  v  with  en+l  •  v  =  t>n+i  >  0.  In  this  case,  the  error  in 
the  constrained  best  approximation  is  E  =  l/v„+i.  Any  constrained  best 
approximation  gopt  corresponds  to  some  optimal  vector  v  under  the  map 
9  —  <*«/«  with  a,  :=  -V{/vn+ j  and  vice  versa:  any  such  optimal  vector 

v  yields  an  optimal  solution  gopt  by  the  same  map. 

The  proof  of  this  result  is  straightforward,  and  is  in  fact  a  direct  application 
of  a  minimum  norm  duality  theorem  of  Deutsch  and  Maserick  (1967)  and  found  in 
Luenberger  (1969),  page  119,  and  thus  will  not  be  given  here. 


2.  A  2  phase  algorithm 

For  convenience  in  what  follows,  the  supporting  normal  vectors  appearing  in  C  are 
renamed  and  reindexed,  omitting  any  that  are  0n+i,  i-e-,  C  -  {x  £  Rn+i  |  n(p)  •  x  < 
0 ,  p  6  J*}.  Also,  set  C  n  H  :=  Ch- 


2.1  Phase  1 

Problem  Vh  can  be  efficiently  solved  by  an  interior  point  algorithm  to  be  described 
in  the  following  section  provided  K  fl  Ch  has  interior  in  Rn+1.  As  K  always  has 
interior,  int(ff  fl  H)  is  empty  iff  int(C//)  =  0.  In  terms  of  the  constraints  in  Vc,  this 
situation  obtains  when  either  inteTpolatory  conditions  are  imposed  (not  satisfied  by 
all  g  €  V),  or  when  a  collection  of  “one-sided”  constraints  has  “ganged  up”  to  form 
a  “  generalized”  interpolation  condition  or  conditions.  As  the  solution  of  general 
problems  is  the  goal,  Phase  2  is  preceded  by  an  analysis  of  the  cone  Ch  determining 
the  dimension  of  its  relative  interior,  followed,  if  necessary,  by  a  suitable  introduction 
of  a  reduced  set  of  basis  functions  {^i, . . ., g^},  h  <  n,  with  linear  span  V  C  C,  such 
that  all  interpolation  conditions  are  automatically  satisfied  by  any  g  £  V.  Hence, 
by  adopting  V  as  the  approximating  space,  the  resulting  cone  Ch  will  have  interior 
in  the  reduced  dimension  space  R*+l. 

It  turns  out  that  the  key  to  Phase  1  is  determining  whether  0„+i  is  in  the  convex 
hull  of  <?n+i  U  {n(p)}p€j».  If  so,  one  determines  the  set  L  of  all  such  vectors  which 
are  involved  in  nontrivial  convex  combinations  yielding  0„+i-  The  resulting  set  L 
has  orthogonal  compliment  Lx  containing  Ch-  In  essence,  the  set  L  corresponds 
precisely  to  the  constraints  (if  any)  which  have  “ganged  up”  in  Vc  to  reduce  the 
degrees  of  freedom  in  the  approximating  space.  That  is,  a  basis  for  Lx  yields,  under 
the  inverse  of  the  homogeneous  coordinate  transcription,  a  reduced  basis  {pi, . . . 
to  work  in,  with  dim(Lx)  =  n,  or  shows,  in  case  n  =  0,  that  the  problem  Vc  is 


1 

L 


I 

overconstrained.  Fortunately,  such  a  task  is  efficiently  undertaken;  see  Bosworth 
(1988)  for  a  detailed  algorithm  description.  Moreover,  if  L  is  empty,  then  the  just 
cited  algorithm  provides  one  with  a  vector  u  £  Ch,  separating  0n+i  from  the  convex 

hull  of  en+1  U  {n(p)}p€j-.  • 


2.2  Phase  2:  the  tunnelling  algorithm 

A  description  is  now  given  of  the  upper  level  structure  of  the  actual  Phase  2  semi- 
infinite  programming  algorithm  solving  the  problem  A  detailed  description  of 
Phase  2  can  be  found  in  Bosworth  (1984)  .  Note  that  on  successful  completion 
of  Phase  1,  a  vector  u  pointing  into  the  interior  of  K  n  Ch  is  available,  and  that 
0„+i  G  K  n  C.  Note  also  that  in  the  course  of  Phase  1,  the  dimension  n  of  the 
problem  has  been  suitably  redefined  if  necessary.  Initialize  :=  0n+1. 

2.2.1  Shooting 

From  the  point  f^,  “shoot”  in  the  positive  u  direction  to  sx  E  d(K  DC).  If  no  such 
point  exists,  then  u  satisfies  Result  1,  condition  2,  and  the  algorithm  terminates. 
If  sx  #  dK,  then  scale  sx  positively  until  it  reaches  dK.  Again,  if  no  such  point 
exists,  then  the  direction  vector  sx  satisfies  Result  1,  condition  2,  and  the  algorithm 
terminates. 

At  sxj  :=  sx  compute  a  unit  outer  supporting  normal  snj  to  dK ,  using  the 
subgradient  d||x  •  /j|  at  sxj.  If  more  than  one  normal  exists  (nonsmooth  norm), 
select  one  with  maximal  n  +  1“  component.  If  the  normal  sh1  =  en+J,  then  sxj  is 
the  optimal  solution  to  P*,  Result  1,  case  3,  and  the  algorithm  terminates. 

2.2.2  Webbing 

Set  M  :=  {x  €  •ftn+1|  en+1  •  x  =  e„+i  sxi}  and  N  :=  {x  G  jRti+1|  en+l  -x  =  0}.  I.e., 
M  =  N  +  sii . 

A  collection  of  n  +  1  supporting  halfspaces  to  the  set  K  DC  with  support  points 
on  d(KnC)DM  is  constructed  by  means  of  a  n  stage  recursion  (using  the  shooting 
technique  of  the  previous  subsection),  the  aim  being  to  “equidistribute”  the  support 
points  on  d(K  fl  C).  These  halfspaces  will  be  used  in  the  following  stage  of  the 
algorithm  to  define  a  minimal  linear  program  approximating 

The  fcth  stage  of  the  recursion  (fc  =  2, . . . ,  n  +  1)  is  “roughly"  the  following: 

*cm,k- 1  :==  jfcZY  s~*i 

Hi  ■-  proj/v  shi  ,  i=  1,...,*-  1, 

define  the  fcth  search  direction  tf*  by: 

u*  •  n,  =  — 1  ,  *  =  1,  —  ,A  —  1 

u*  G  Iin{ni,...,n*_i}, 

and  then  the  supporting  vector  sx*  and  outer  supporting  normal  sn*  to  K  fl  C 
are  found  by  “shooting”  from  in  the  positive  if*  direction.  However,  here 
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sn,  is  allowed  to  be  a  supporting  normal  to  the  cone  C.  (If  the  vectors 
are  dependent,  randomly  chosen  vectors  in  N  are  added  to  the  set  until  a  well 
defined  direction  u*  can  be  obtained  from  the  above  relations,  hence  the  adjective 
“roughly”.) 

By  construction,  K  nC  C  {i  E  i?n+1|  sn,  •  i  <  si,  •  si,}  for  t  =  1,--  -  ,k.  At 
the  conclusion  of  the  recursion,  set  i^  :=  si,. 

2.2.3  Analysis  of  a  minimal  linear  program 
Consider  the  simple  linear  program: 

P  f  max  en+\  ■  x 
m  p  1  shi  ■  x  <  sn,  •  si,  i  =  1 , . . . ,  n  +  1 

By  construction,  K  C\C  is  within  the  feasible  region  defined  by  Vmip,  and  hence  an 
optimal  value  of  VmlP  provides  an  upper  bound  on  the  optimal  value  of  Vh-  Moreover, 
Vmip  is  trivial  to  analyze  (one  “Simplex  Pivot”).  Since  VmiP  can  be  considered  to 
be  a  “coarse”  discretization  of  the  semiinfinite  program  Vh ,  it  will  be  reasonable  to 
use  any  information  obtained  from  the  optimality  analysis  of  Vmip  as  approximating 
the  situation  in  Vh- 

Four  cases  may  obtain  in  Vmip: 

1.  There  exists  a  unique  optimal  solution  vmlp-  Then  the  feasible  region  in 
Vmip  intersected  with  H  is  a  bounded  polytope  containing  K  n  Cm-  The 
solution  Vmip  can  be  considered  an  approximate  solution  to  Vh-  Define  a 
new  search  direction  for  the  “shooting”  stage,  u  :=  vmip  -  xm. 

2.  There  exists  infinitely  many  optimal  solutions  wmip,  all  with  the  same  n+  1st 
component.  Define  a  new  search  direction  u  :=  vmip  —  Xcm  ,  where  vmip  is  the 
closest  optimal  solution  to  !<*„.  It  can  be  easily  shown  that  this  selection 
of  u  has  nonnegative  n  +  1st  component. 

3.  The  system  of  equalities  Ax  =  b  with  row,  A  :=  sn,  ,  6,  :=  sn,  •  si,  ,  i  — 
l,...,n+  1  is  singular,  but  has  vectors  w  G  Kern  (A)  with  positive  n  +  l8t 
component.  Select  the  unit  vector  u  E  Kern(A)  with  maximal  n  +  1st 
component  as  the  new  shooting  direction. 

4.  The  system  Ai  =  b  is  nonsingular,  but  Vmip  has  unbounded  objective  values. 
Then  it  can  be  shown  that  there  exists  a  ray  of  unbounded  feasible 
solutions  to  Vmip  with  unit  direction  vector  u  having  maximal  positive  n+l*‘ 
component.  This  is  the  new  shooting  direction. 

In  each  of  the  4  cases,  a  vector  u  is  obtained,  which  heuristicaily  points  from 
Xcm  G  A'  n  C  in  a  direction  of  “ascent”  for  Vh- 

2.2.4  Convergence 

If,  in  case  1  or  2  above,  vmip  -  xm  is  “sufficiently  small”,  then  one  can  conclude 
convergence.  If  en+i  •  xm  <  (1/tol),  where  tol  is  an  upper  bound  on  the  error 
needed  in  a  near- best  approximation,  then  one  concludes  convergence.  In  either 
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case,  to  obtain  the  final  v  to  use  in  computing  gba,  shoot  one  last  time  from  xc 
the  u  direction  to  v  £  d(K  n  C). 

Else,  one  returns  to  the  shooting  stage,  and  continues  with  the  algorithm. 


in 


3.  Conclusions 

Linear  convergence  of  the  algorithm  can  be  concluded  provided  Phase  1  is  suc¬ 
cessfully  completed;  see  Bosworth  (1984).  That  is,  the  sequence  of  points  {£„,} 
converges  in  objective  value,  from  below,  to  the  optimal  value  for  Vh  at  least  at 
a  linear  rate.  If  is  in  situation  3  of  Result  1,  then  one  is  also  guaranteed  the 
existence  of  subsequences  of  {i'ctn}  and  {rm/p}  having  convergence  in  Rn+1  to  an 
optimal  solution  v  of  Vh  (different  subsequences  can  have  different  limits;  however, 
all  tend  to  the  same  objective  value).  In  polyhedral  norm  situations,  with  a  finite 
set  of  constraints,  the  algorithm  converges  in  a  finite  number  of  steps,  often  out¬ 
performing  the  Dual  Simplex  Algorithm  of  linear  programming,  due  to  its  ability  to 
bypass  in  one  step  several  pivots  taken  by  the  Dual  Simplex  Algorithm.  (The  Dual 
Simplex  Algorithm  computes  a  sequence  of  infeasible  solutions,  converging  to  the 
optimal  solution  of  Vh,  similar  to  the  solutions  {t7m;p}  computed  here.) 

Numerical  results  for  the  case  of  uniform  norm  shape  constrained  approximation 
problems  have  appeared  in  Bosworth  (1987)  .  Applications  to  the  lx  and  other  spe¬ 
cially  tailored  norms  will  appear  shortly.  One  final  observation  is  that  the  viability 
of  the  algorithm  appears  most  strongly  in  non  lj  settings.  In  constrained  l?  settings, 
the  code  of  Lawson  and  Hanson  (1974)  is  to  be  recommended. 
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Abstract  We  consider  a  formula  for  smoothing  noisy  data  in  which  the  variances 
are  different.  The  formula  is  obtained  by  a  local  weighted  approximation.  The 
weight  is  a  function  of  a  smoothing  parameter  a,  which  varies  with  the  point. 
Then  a  diagnostic  band  for  data  analysis  is  obtained  :  this  determines  whether 
an  initial  and  computationally  quick  choice  of  a  constant  for  all  the  data  allows  a 
homogeneous  accuracy  of  approximation  on  the  whole  domain.  In  the  case  when 
it  is  necessary  for  a  to  be  variable,  a  fast  algorithm  to  evaluate  a  is  proposed. 

Key  words:  Inference  region,  Weighted  local  approximation,  Smoothing,  Vari¬ 
able  smoothing  parameter,  Data  analysis. 


1.  Introduction 

In  the  problem  of  approximating  a  function  from  a  sample  of  function  values,  it  is 
of  interest  to  know  the  reliability  achieved  with  the  chosen  approximation  method. 
In  other  words  what  is  the  probability  that  the  approximating  function  fits  the 
unknown  function  to  a  specified  accuracy? 

On  this  subject  we  refer  to  the  work  by  Wahba  (1983),  who,  on  the  basis  of  a 
Bayesian  model  (that  is  to  say  the  region  is  that  within  which  the  approximating 
function  can  fall  with  that  sampling)  for  spline  functions,  assuming  that  the  noise 
variance  is  constant  determines  an  inference  region  with  constant  band.  A  later 
work  by  Silverman  (1985)  generalizes  the  work  by  Wahba  by  considering  more 
general  data  configurations  and  noise  characterized  by  variance  a1  =  <r2(x)  as  a 
function  of  the  abscissa  x.  The  band  obtained  varies  according  to  the  data  location 
density  and  variance. 
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Muller  and  Stadtmuller  (1987)  study  a  method  with  a  localized  smoothing 
parameter  based  on  estimates  of  the  higher  derivatives  of  /(x). 

In  this  note,  on  the  basis  of  an  approximation  method  with  constant  window, 
we  construct  an  inference  region  for  the  unknown  function  /(x)  (in  other  words 
/(x)  falls  in  the  region  derived  from  that  sampling  with  given  probability)  by 
using  samples  of  random  variables  /r(x)  r  =  1,...,/  /  >  1  and  finite,  with 
E(fr(x))  =  f(x)  and  covariance  matrix  cr*t  =  6rta 2. 

The  method,  which  is  computationally  simple,  has  variable  band  and  indicates 
the  regions  where  more  accuracy  is  needed  when  constructing  the  approximating 
function. 

Then  a  computationally  straightforward  method  is  suggested  to  improve  the 
approximation  locally,  by  modifying  the  local  smoothing  parameter. 


2.  Approximation  formula 


Let  a  set  J  =  {x,-,  /(x;)}^  be  given  on  a  domain  D  =  [a,b]  C  R  ■  An  approximation 
to  f(x)  is  required. 

We  assume  that  the  function  values  /(x<)  are  the  result  of  sampling  from  l 
populations  with  sample  sizes  ni,n2, .. .  ,nj  ,  respectively,  each  one  according  to  a 
uniform  distribution  (for  simplicity)  and  nT  =  N  and  such  that 

E(fr(x))  =  /(X), 

<  =  ST.al 

For  simplicity,  in  the  following  it  is  assumed  that  nT  =  n  for  r  =  1 , . . . ,  /. 

Let  us  consider  the  method  studied  in  Bozzini  and  Lenarduzzi  (1988a)  to  ap¬ 
proximate  the  unknown  function  /(x). 

Precisely,  after  indicating  with  x,  a  generic  assigned  point  ,  take  a  neighbour¬ 
hood  I  =  [x,  —  R,  Xj  +  jR]  and  determine  the  constant  function  c,  by  the  weighted 
least  squares  method: 

min  £  wjQ(c- f(xj))2 

*,€/ 

where  the  weights  Wja  have  the  following  expression: 

w  =  /  W+Z&  if  d<R’ 

ia  1  0  if  d  >  R, 


with  cP  =  dist2(xi,Xj)  the  Euclidean  distance  and  a  a  positive  smoothing  param¬ 
eter. 

Then  the  approximating  function 


m(x) 


(1) 


is  constructed. 

Here  the  ^<(i)  are  smooth  functions  with  support  —  +  with  R\  <  R 

and  such  that  m(x,)  =  c,-. 

This  approximating  function  depends  on  a;  in  particular,  as  a  — ►  0  the  function 
m(x)  tends  to  an  interpolating  function,  while,  as  a  — >  oo  m(x)  tends  to  the 
function  in  which  the  constants  c,  are  solution  of  the  weighted  least  squares  problem 
with  weights  u>,r  = 


3.  Inference  region 

Let  us  assume  we  work  in  asymptotic  conditions  (that  is  to  say  with  N  — ►  no,  R  — > 
0,  NR  — ►  oo)  and  that  we  also  have  information  beyond  the  boundary  of  D. 

After  constructing  the  function  fh{x)  on  the  basis  of  (1)  we  determine  a  com¬ 
putable  inference  region. 

From  the  Markov  inequality  (see  Ross  (1980)  ),  one  gets  for  i  =  1, . . . ,  N: 

Prob{\  f{n)  -  m(x,)  |<  fc{£e[/(x,)  -  m(x,)]2}*}  >  1  -  p. 

The  expected  mean  squared  error  Z?e[/(x,)  -  m  (x,)]2,  which  will  be  called 
EMSEi  in  what  follows,  has  the  expression: 

EMSEi  =  {Ee[f(Xi)  -  m(xj)]}2  +  var(m(x,)) 

and,  of  course,  it  is  a  function  of  the  parameter  a. 

In  Bozzini  and  Lenarduzzi  (1988a)  it  was  proved  that  there  is  one  and  only  one 
value  a*  of  a  that  minimizes  EMSEi(a ): 

EMSEi(a*)  =  mina>0EMSEi(a) 

and  such  a  value  falls  in(0,  R2)  or  in  [f?2,oo)  depending  on  whether  the  ratio 


R*fj2)2 

[2n«] 

is  greater  or  less  than  one,  (here  6  =  (£*  ^r)-1). 

Furthermore,  using  the  hypothesis  that  /(x)  can  be  developed  as  a  power  series, 
the  principal  part  of  EMSEi{a*)  has  the  following  expression: 

C 
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EMSEi(a*  )  ~  ^ R2a*  +  - 

2  riy/a*n 
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if  at*  £  (0,  R 2)  and 


L 

I 

L 
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! 

I 
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EMSEi(a *  ) 


-  A)  +  Au  -  0) 


if  at*  €  [ R 2,  oo) 

It  can  be  verified  analytically  that  an  upper  bound  to  the  principal  part  of 
EMSEi(a*)  is: 


1.56p,,«  ), 


(2) 


where  p,-,(a)  is  the  asymptotic  value  of  the  weight  Wia/  Y1  wja  (index  of  influence 
of  the  i-th  datum)  divided  by  ^  and  associated  to  the  value  c<  at  the  point  x,. 
Consider  the  random  variable 


= 


(3) 


with 

i 

then: 


Ee{e)=8+m™^-6, 


zN,p,i 


fSl  -Pa) 

If  one  assumes  that  the  second  derivative  does  not  change  too  much  on  I  then 
AMSEi(ai)  =  Ee  ^(/(<)  -  rh(t)fdt  =  2  R  EMSE .-(a,-)  +  0{F?) 

In  optimal  conditions  (that  is  to  say  X  R2),  it  has  been  proved  in 

Bozzini  and  Lenarduzzi  (1988a)that  EMSEi(a*)  =  0(n~$)  and  pa  =  0(n~s);  so: 


Ee(0(<*l  ))  =  «  +  0(n-*). 


(4) 


As  can  be  seen  by  developing  the  calculations,  one  has: 

var(0(a*  ))  =  0(n~>) 

so  that  0(at*  )  is  consistent  estimator  for  6. 

From  (4)  it  follows  that  the  principal  part  of  1.5 pu(a*)Ee(0(a*  ))  is  given  by 
1.569,-, (a*).  On  the  basis  of  (2)  and  (3)  it  follows  that  an  inference  region  which 
is  computable  for  /(x)  at  the  probability  level  (1  —  p-)  is  given  by: 

"»  Q*  (z)  ~  k(l.50  *  p„(af  ))»  <  /(x)  <  m  *  (x)  +  F(1.50  *  p,-,(of  ))*.  (5) 

In  order  to  make  the  method  more  efficient,  in  the  case  when  the  second  deriva¬ 
tive  does  not  change  too  much  on  the  domain,  it  is  preferable  to  calculate  a  constant 
value  of  a  for  all  x<  €  J  ■ 
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This  is  possible  by  calculating  the  value  a ^  which  minimizes  IMSE  = 
EeJD(f(t)  —  m(t))7dt  (see  Bozzini  and  Lenarduzzi  (1988a)). 

In  this  case  (5)  becomes: 

V  ’  (6) 
rh  atopt  ( x )  -  k(l.50aopt  Pa^t  ))*  <  /(*)  <  ™  a<ypt  (x)  +  fc(1.50Qop«  P«(aopt  )))*• 

In  the  case  in  which,  for  some  Xj,  ( / »  (f\?)2  =  /«  ^77(0^  then,  as  de¬ 
rived  in  Bozzini  and  Lenarduzzi  (1988a),  both  EMSEi(a *  )  and  E(6(aopt  ))p„(aopt ) 
increase  because  of  the  bias;  therefore  this  band  is  wider  than  that  for  the  other 
points. 

Remark: In  the  case  when  there  is  only  one  variance,  that  is  to  say  a7  =  a7  , 
r  =  1, . . . ,  /,  the  above  remains  true  provided  that  one  substitutes  Wja  = 
with  Wja  =  and  6  with  a7. 


4.  Variable  smoothing  parameter 

Let  us  consider  a  smooth  function  /(x)  which  has  peaks  at  some  points.  Such  a 
function  therefore  has  a  second  derivative  f$2  which  is  almost  constant  and  given 
by  /ak  but  at  the  point  x„  /j2)2  >  /$)2. 

The  parameter  a, which  is  optimal  on  the  whole  interval  [a,  6],  falls  in  [R2 , 00)  as 
the  method  constructs  an  approximation  function  in  the  least  squares  sense  which 
removes  the  noise  and  provides  a  smooth  function.  On  the  contrary,  the  optimal 
value  of  a  at  the  point  Xj  falls  in  the  interval  (0,  R7)  in  order  to  give  a  function 
which  is  locally  nearer  the  interpolating  function. 

Consider  the  random  variable: 


Z  = 


£?#(/< -m,)2 


Its  expected  value  is 


E(Z)  =  6+  _/M5£t  -  6-  Ep" 


E(1  -  Pa)  E(1  -PiiY 

In  the  case  we  are  studying,  the  optimal  value  of  a  for  IMSE  is  and  therefore: 

E(Zopt)  =  6(l  +  kin-i). 

Assuming  a*  x  one  gets: 

£(0(a*  ))  =  6(1  +  k2n'&), 

from  which 


t 


« 


« 


Ratio  = 


E(Z<»t) 

E'M*  )) 


~  1  —  kjn  s-« 


On  the  contrary,  when  one  constructs  locally  the  function  m  (x)  using  the  value 
a „pe  instead  of  a*  : 


Ee(0(aopt  ))  ~  fi(l  + 


from  which 


ifatto  = 


E(Z^) 

EeWaopt  )) 


~  1  —  k3n  v"5-*) 


(here  the  kj  are  positive  constants). 

It  is  now  evident  that  the  band,  calculated  by  using  0(a )>  is  no  longer  an  in¬ 
ference  band  but  only  a  tool  for  data  analysis  :  in  fact  on  one  hand  E(0(a*  ))p„(a*  ) 
and  EMSEi(a*  )  are  of  the  order  of  n-*^,  but  on  the  other  E(0(aopt  ))p,-,(aopt  ) 
is  of  the  order  of  n”s . 

In  order  to  obtain  a  better  estimate  of  the  function  on  that  interval  and  more 
reliability  it  is  appropriate  to  modify  locally  the  value  of  a. 

This  can  be  done  according  to  the  following  law: 


®  —  &opt 


*s{ 


zppt  ^ 

r 


where  <7  is  an  increasing  function  of  its  argument  and  5(1)  =  1. 


« 


« 
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5.  Numerical  results 


We  refer  to  two  cases  from  the  references. 

For  the  estimates  of  a ^  and  cr2  see  Bozzini  and  Lenarduzzi  (1988b)  . 
Case  1:  test  function  from  Muller  and  Stadtmuller  (1987): 

g2(x)  =  h( 0.25,0.05)  +  h(0.5,0.1), 


where  h  is  a  Gaussian  density  N  =  100,  k  =  4.48,  R  =  .039,  Oj  =  0.01,  a2  =  0.02. 

The  function  is  smooth,  so  a  good  approximation  accuracy  is  reached  and  the 
confidence  band  is  uniform  enough,  by  using  (6).  The  results  are  presented  in  figure 
1. 

Case  2:  test  function  number  4  in  Wahba  (1983): 


/(*)  = 


r  0, 

36(*  -  i), 
36(§  -  <), 
l  0, 


0<x<i, 
1  <  t  <  ± 

l<t<l' 
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normal  noise  <r  =  0.02,  iV  =  100,  fc  =  4.48,  R  =  .049,  a  =  0.01. 

The  function  presents  discontinuity  points  in  the  first  derivative. 

When  using  (6)  to  construct  the  inference  band,  one  observes  an  enlargement 
of  the  band  near  the  discontinuity  points  (see  figure  2). 

In  figure  3  can  be  seen  the  approximation  with  the  global  a  and  the  data. 

To  obtain  better  accuracy  it  is  possible  to  use  the  method  described  in  section 
4,  with  the  following  function: 

g(ratio)  —  (ratio)10. 

The  resulting  band  is  presented  in  figure  4. 


Figure  1 


Figure  2 
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5.  Complex  Approximation 


NUMERICAL  METHODS  FOR  CHEBYSHEV  APPROXIMATION  OF  COMPLEX-VALUED 


FUNCTIONS 

G  A  WATSON 

Department  of  Mathematics  and  Computer  Science 
University  of  Dundee 


Abstract  This  paper  is  concerned  with  die  problem  of  best  Chebyshev  approximation  of  a  given  com¬ 
plex  function  on  a  contour  of  the  complex  plane.  The  emphasis  is  on  the  provision  of  methods  which 
can  combine  global  convergence  with  rapid  local  convergence,  and  some  different  approaches  are 
examined,  with  particular  attention  being  paid  to  the  special  case  when  the  approximating  function  is 
linear.  Examples  are  used  to  illustrate  various  situations  which  can  arise,  and  numerical  results  are 
presented  to  demonstrate  the  perfomance  of  the  methods  on  a  variety  of  linear  problems. 

Keywords:  Chebyshev  approximation,  complex  functions,  algorithms,  global  convergence. 

1.  Introduction 

Let  T  be  a  contour  of  the  complex  plane  C,  and  let  r:CmxT-*C  be  an  analytic  function.  Then  of 
interest  here  is  the  problem: 

find  zeCm to  minimise  ||r(*„  )ll .  (!) 

where  ||r(a„  )||  =  maxlr(z,w)l,  and  z  =  (zxa2 . *»)TeC".  Such  problems  frequently  arise  from 

w«r 

best  approximation  problems  defined  on  a  simply  connected  region  of  the  complex  plane  with  boundary 
T,  for  the  use  of  the  maximum  modulus  theorem  means  that  attention  may  be  restricted  to  the  boun¬ 
dary.  For  j  =  1,2,...,*,  let  fy(z,w)  denote  the  partial  derivative  of  r  with  respect  to  z,.  By  considering 
the  real  problem  equivalent  to  (1)  (i.e.  with  2n  real  variables  formed  by  taking  the  real  and  imaginary 
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parts  of  z),  the  following  result  may  be  established  (see,  for  example,  Ben-Tal  et  al  (1979)),  giving 
necessary  conditions  for  a  solution  to  (1). 

Theorem  1  Let  zeC*  solve  (1).  Then  there  exist  mUn+l  points  w1,w2,...,w,,  in  T,  with 
lr(z,w;)l  =  ||r(z„  >11,  y=l,2,...,m, 

and  Xj,  j=\X—jn  real  nonnegative  numbers  (not  all  zero)  such  that 

%kjr(z,Wj)Qk(z,wi)  =  0,  *=1,2, 

Remark  For  the  special  case  when  r  is  an  affine  function  of  z,  it  follows  that  the  conditions  of  the 
theorem  are  also  sufficient  for  a  solution,  and  therefore  represent  characterization  conditions  (see  also 
Lorentz(1966)).  Otherwise  the  conditions  may  be  said  to  define  a  stationary  point 

Assume  that  T  can  be  parameterized  by  die  real  number  t€[a,f)]  so  that  w:[a,p]-»r  and 
wj  =  w(tj),  y=l,2,...,m.  Define  fJi^xR  — >1?  by 

f(x,t}=lr(z,w(t))\2,  r€to,p)^eC", 

Lx  1  , 

where  x  =  j  eR  ,  and  the  superscripts  R  and  /  denote  real  and  imaginary  parts  respectively. 

Without  loss  of  generality,  assume  that  are  internal  to  [a$).  Then  if  m  and  p  are  known,  the 

conditions  of  the  theorem  may  be  written  as  the  nonlinear  system  of  equations 


ZWX4)  =  0.  (2a) 

/-i 

l-ZX;  =  0,  (2b) 

f\xjj)  =  0,  j=lX.,p,  (2c) 

f(xjj}-h  =0  ,  y=l,2,...,m,  (2d) 


where  V  denotes  differentiation  with  respect  to  xeR2*  .the  dash  denotes  differentiation  with  respect  to 
t  and  the  fact  is  used  that 

tel'**- 

•  • 

with  ^(+i,fe,  *  *  -  ,4»,)r  .  This  is  a  system  of  2n+m+p+l  real  equations  for  the  2n+m+p+l  real 
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unknowns  xeR^^eR ,  j=l2,../n,  tjeR ,  and  heR,  where  A=)|r(z,.  )||2  at  a  solution  to 

(1). 


2.  A  locally  convergent  method 

The  Jacobian  matrix  of  the  system  (2)  may  be  written  in  the  form 


H  0 
0  0 
A  0 


A7  A 
C? 
D 


Px 

0 


gt 

-T 


(3) 


where 

H  =  £A;V2/(x,  tj  )  e/?2"*2". 


A  =  diag  . Ay}, 

^/(x,  tt) 

Ajt  =  ~  dtdxf  >  k=U,...^». 

a/(x,  t. ) 

Gfi  =  -  ,  j=lZ...jn, 

dxt 


D  =  diag  ),  j=12,..j>). 


D,=  diag  (/'(*.  tj  ),  j= \X-.J> }, 


and  e  denotes  the  vector  in  Rm  all  of  whose  components  are  1.  Notice  that  D  is  negative  semi-definite 
at  a  solution  to  (2)  which  is  also  a  solution  to  (1).  Also 


VV(x,0  =  2 


Re  (iff)  /«(♦£) 
-Im(W)  Re(W) 


+  2 


Re(rT)  Im{rTJ 
Im(rT)  - Re(rT ) 


(4) 


where  r  is  the  Hessian  matrix  of  r  with  respect  to  z. 

The  system  of  equations  defining  the  Newton  step  is  therefore  an  unsymmetric  2n+m+p+l  by 
2n+m+p+\  system  of  equations.  Considerable  improvement  may  be  obtained  if  use  is  made  of  the  fact 
that  Dj  is  zero  at  a  solution  to  (1),  so  that  it  may  be  replaced  by  zero  in  (3)  without  inhibiting  a  second 


order  convergence  rate.  The  system  of  equations  defining  the  increment  vector  may  then  be  written 


H  0  AtA  Gt 

Sx 

-GtX 

0  0  0  -eT 

8 h 

XTt- 1 

A  0  D  0 

St 

-4 

G  -e  0  0 

.sxj 

Ae-f 

where  i  e  Rr  denotes  the  vector  with  jth  component  /'(x,  tj),  j= and  f  e  Rm  denotes  the 
vector  with  jth  component  /  (x,  ry  ),  j=\2,-jn.  This  system  of  equations  may  readily  be  broken  down 
into  smaller  systems.  Indeed  the  following  algorithm  may  be  used  which  first  computes  a  first  order 
estimate  of  X  ( an  approximation  to  \+8k  satisfying  (5))  and  then  provides  approximations  to  5x,  5A ,  5t 
satisfying  (5).  When  used  as  increment  vectors,  a  second  order  convergence  rate  is  not  inhibited  (  see. 
for  example  Watson(198S)>. 


Algorithm  1 


Step  1  Let  Cr* 


=  [Y  :Z  ] 


R 

0. 


=  YR, 


where  [T  :  Z]  is  (2n+l)x(2n+l)  orthogonal  and  R  is  mxm  upper  triangular,  nonsingular  if  C  has  full 
rank  (a  necessary  condition  for  the  system  (5)  to  be  nonsingular). 

Step  2  Determine  X  satisfying  R  X  =  -K^e^+i  (  ei(+l  is  the  (2/1+1)*  co-ordinate  vector)  and  scale  X  so 
that  XTe  =  1. 

Step  3  Solve  RTdt  =  Ae-f. 

Step  4  Form  W  =  H  -  ATAD~lA  ,  and  solve 


Z[WZ,d2  =  Z](ATAD~l(  -  WTidj  )  -  x2. 


where  Y\Z\  denote  the  first  2 n  rows  of  Y  jL  respectively,  and  zj  denotes  the  last  row  of  Z. 

Step  5  Form  [jj^J  =  Tdi  +  Zd2  and  set  x  =  x+6x,  h  =  A+5/r 

Step  6  Form  6t  =  -D~l(f+A  Sx)  and  (unless  convergence)  set  t  =  t+6t  and  return  to  Step  1. 

This  algorithm  will  be  dominated  by  the  calculation  of  die  QR  factorization  of  CT  in  Step  1.  It 
can  converge  to  a  solution  of  (1)  if  provided  with  good  initial  approximations,  and  with  the  correct 
information  about  m  and  p .  Of  course  even  if  convergence  is  obtained  it  has  still  to  be  confirmed  that 
h  =\\f  ||  ,  and  this  is  a  nontrivial  problem  in  itself.  Its  solution  is  connected  with  a  modification  of  this 
method  in  which  the  vector  t  at  each  iteration  is  defined  to  be  the  appropriate  set  of  m  local  maxima  of 
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the  current  /  in  [a^]:  this  calculation  can  be  done  as  a  preliminary  step,  with  Step  6  of  course  no 
longer  required.  Provided  that  D  is  nonsingular,  and  /  (xjj  )*OJ  =p+\„.jn  the  implicit  function 
theorem  can  be  used  to  give  t  as  a  differentiable  function  of  x,  and  the  Newton  step  (in  the  variables 
x,h  A)  derived.  The  analogue  of  Algorithm  1  is  then  obtained  by  introducing  Step  0  as  described,  drop¬ 
ping  Step  6  (testing  far  convergence  in  Step  5)  and  setting  (  to  zero  in  Step  4:  a  second  order  conver¬ 
gence  rate  can  again  be  established. 

The  following  examples  all  use  Algorithm  1  in  its  original  form. 

Example  1  (Glashoff  and  Roleff(1981))  Consider  the  approximation  of  w*  by  a  complex  polynomial  of 
degree  7  on  the  rectangle  [-2,2]  x  [-1,1]  in  the  complex  plane.  Because  of  symmetry,  it  is  sufficient  to 
consider  the  approximation  of  w*  by  a  real  linear  combination  of  1,m>2,w4,w6  on 


J  2+it,  ( 

|(3-t)+i. 


OStSl 
,  1STS3 


The  fact  that  the  coefficients  are  real  means  that  HA  and  G  may  all  be  reduced  in  size  with  conse¬ 
quent  reduction  in  the  computation  involved  in  Algorithm  1.  Taking  the  initial  approximation 
z  =  {(-15.19822,0),  (46.28396,0),  (-31.88876,0),  (7.748537,0)]  ,  t  =  (0.325,  1.8,  1.0,  3.0),  with  m  =  4 
and  p  =  2  (interpreting  t3  =  1.0  as  an  end  point),  and  h  =  10428.39,  the  performance  of  the  algorithm 
is  summarized  in  Table  1.  The  column  headed  k  gives  the  iteration  number  and  5  gives  the  maximum 
modulus  component  of  8x,5 h  and  St  The  CPU  time  (DEC  10  single  precision)  was  0.78  seconds. 
(Notice  that  the  size  of  the  matrix  (3)  in  this  case  is  11  x  11,  whereas  the  corresponding  Jacobian 
matrix  in  the  method  used  by  Glashoff  and  Roleff  is  15  x  15). 


5 

B 

0.149281 

0.315838 

1.794251 

0.479269 

0.316292 

1.795387 

0.008201 

0.316291 

1.795388 

0.000025 

0.316291 

1.795388 

Table  1 


Example  2  Consider  the  approximation  of  l/(w  -  (2+i ))  by  a  polynomial  of  degree  7  on  the  part  of  the 
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unit  circle  defined  by  0€[O,3ji/2}.  In  this  case  m= 9  and  p=l.  Taking  the  initial  approximation  z  = 
((-0.4, 0.2),  (0.1202,0.1602)  ,(-0.01653,0.08797),  (0.01073,0.03786),  ,(0.001226,0.01225). 
(0.008144,0.002251),  (0.004343,-0.00308),  (0.001954,-0.000459)},  t=  (0,0.3436  ,0.8467,  1.399, 
2.0985,  2.815,  3.607,  4.3196,  3tc/2)  and  /i=0,  the  algorithm  performed  as  shown  in  Table  2.  The  CPU 
time  was  2.71  seconds. 


k 

5 

hxior7 

1 

0.04765 

4.0788 

2 

0.04300 

4.3852 

3 

0.01111 

4.44382 

4 

0.000362 

4.44747 

5 

0.000005 

4.44755 

Table  2 


4 


4 


4 
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Example  3  This  is  the  approximation  of  exp(w)  on  the  part  of  the  unit  circle  defined  by  0e  [0,rt]  by  a 
rational  function  farmed  by  the  quotient  of  two  complex  polynomials  of  degree  2:  the  rational  function 
is  normalized  by  choosing  the  constant  in  the  denominator  to  be  1.  In  this  case  m  -  6  and  p  =  4.  Tak¬ 
ing  the  initial  approximation  z  =  {(1.0066,0.0003384),  (0.51337,0.039861),  (0.085256,  0.01995), 
(-0.48537,  0.037976),  (0.073634,  -0.015799)},  (with  the  natural  ordering  of  the  coefficients),  t  =  {  0, 
7a,  22a,  42a,  65a,  it  },  where  a  =Jt/81,  and  h  =0.0,  the  algorithm  performed  as  summarized  in  Table  3. 


k 

6 

hxior7 

1 

0.02148 

1.69225 

2 

0.01872 

1.77062 

3 

0.000452 

1.77476 

4 

0.000002 

1.77477 

Table  3 
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The  question  remains:  how  can  information  about  m  and  p ,  and  also  good  approximations  to  the 
unknowns,  be  provided?  It  should  be  emphasised  that  extremely  good  initial  approximations  may  be 
required:  for  instance  in  Example  3,  small  perturbations  of  the  initial  values  can  lead  to  divergence.  In 
particular,  very  good  approximations  to  t  are  often  necessary:  there  is  more  about  this  later  on.  Perhaps 
the  most  obvious  way  of  finding  initial  data  is  from  the  solution  (or  approximate  solution)  of  a  discreti¬ 
zation  of  (1),  where  T  is  replaced  by  a  finite  discrete  subset,  and  this  gives  rise  to  a  class  of  two-phase 
methnds  Algorithms  of  this  type  have  been  suggested  for  linear  problems  (when  r  is  is  an  affine  func¬ 
tion  of  z)  by,  for  example,  Glashoff  and  Roleff  (1981),  Grothkopf  and  Opfer  (1982),  Opfer  (1982)  and 
Mason  and  Opfer<1986),  based  on  the  formulation  of  the  discrete  problem  as  a  semi-infinite  linear  pro¬ 
gramming  problem.  Linear  programming  methods  may  be  used  to  approximately  solve  this  problem, 
and  advantage  can  be  taken  of  the  special  structure  to  make  this  efficient  (see,  for  example,  Streit 
(1983,  1986),  Streit  and  Nuttall  (1982,1983)).  Notice  that  this  procedure  requires  a  double  discretization 
of  the  original  problem  (1). 

An  alternative  first  phase  procedure  for  the  linear  problem  is  suggested  by  Watson(1988)  where 
the  discretization  of  problem  (1)  is  (directly)  solved  as  a  nonlinear  programming  problem.  Advantage  is 
taken  of  the  special  form  of  the  problem  so  that  a  second  order  convergent  method  is  obtained,  and 
computational  efficiency  is  achieved  by  a  procedure  which  restricts  attention  to  small  subsets  of  the  ori¬ 
ginal  set  of  discrete  points.  This  method  may  be  adapted  to  apply  to  nonlinear  problems,  although  since 
the  convex  nature  of  the  problem  is  lost,  the  provision  of  a  satisfactory  algorithm  represents  a  nontrivial 
modification  and  the  implementation  details  have  still  to  be  resolved.  Some  success  has  been  obtained 
through  the  use  of  a  Gauss-Newton  type  of  approach  (the  simple  device  of  ignoring  the  second  term  on 
the  right  hand  side  of  (4)),  and  this  was  used  to  generate  the  initial  data  for  Example  3. 

Of  course  both  linear  and  nonlinear  discretized  problems  are  examples  of  nonlinear  minimax 
optimization  problems,  and  standard  methods  for  such  problems  are  available.  The  main  computational 
effort  is  expended  in  identifying  the  correct  index  set  on  which  a  solution  is  defined  (analogous  to  the 
points  wx,wv  ...  ,wm  of  Theorem  1),  and  most  methods  are  active  set  methods  which  iterate  towards  a 
correct  index  set  through  equality  constrained  quadratic  programming  subproblems  defined  on  estimates 
of  this  set.  Recent  developments  have  been  concerned  with  making  better  use  of  the  special  structure 
and  solution  characterization  of  minimax  problems,  and  for  example  a  method  having  this  goal  is  given 
by  Yuying  Li(  1988).  A  particular  feature  of  the  problems  considered  here  is  that  fine  discretizations 
(and  therefore  a  large  number  of  points)  may  be  necessary,  and  the  relative  effectiveness  of  different 
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approaches  has  yet  to  be  studied.  In  particular,  the  treatment  of  nonlinear  problems  of  the  type  (1)  and 
their  discretizations  has  received  little  attention  (with  the  exception  of  some  rational  approximation 
problems)  and  so  is  not  something  that  can  be  properly  dealt  with  here.  For  the  rest  of  this  paper, 
therefore,  although  much  of  the  theory  carries  over  (or  may  be  modified  to  apply)  to  the  nonlinear  case, 
attention  will  be  confined  to  the  treatment  of  the  linear  problem  for  which 

r(l,  W)=  £Z;0;(W)  ~  F(W),  W  €  T,  (6) 

y-i 

where  F :r-»C  ,$t  :T~>C ,  j= l,2,...,rt  are  analytic  functions.  Perhaps  the  most  important  consequence  of 
this  is  that  V2/  (x,  t )  is  given  by  the  first  term  on  the  right  hand  side  of  (4)  and  so  is  positive  semi- 
definite;  therefore  so  also  are  H  and  W  provided  that  the  components  of  X  are  non-negative.  H  is  of 
course  independent  of  x. 

3.  Globalization 

The  problem  (1)  may  be  stated  in  the  form 

minimize  h 

subject  to  / (x,  r)  £  k ,  re  [a$],  (7) 

and  a  corresponding  discrete  problem  may  be  written 

minimize  h 

subject  to  /(x,  t{)  £  h,  i= (8) 

where  r,  e  [a,p]  ,  (  =1,2,...^.  Gearly  if  the  discrete  set  contains  the  points  r,  defined  by  Theorem  1  then 
the  minimum  values  of  both  problems  will  be  the  same.  One  approach  to  solving  (7)  is  therefore  to 
iterate  on  a  sequence  of  discrete  subsets  which  in  the  limit  contains  the  appropriate  optimal  set. 

Given  xe/?2*  let  be  the  set  of  local  maxima  (assumed  finite)  of  /  (x,  r)  in  [0,0].  Then 

/'(*,/>)  =  0,  f,€(a$), 

and  provided  that  /'(x,  i,)*0,  t,€(a,0),  /'(x,  tj)  *0,  tj€  [ajJ),  the  implicit  function  theorem  may  be 
applied  to  give  tj,j= as  differentiable  functions  of  x.  Based  on  these  q  local  maxima,  and 
given  X eR<  non-negative  .  defir??  teR<.  GeR**3*,  HeR7”*2*,  WeR7**1*  ,CeR1*(7M+,)  as 
in  Algorithm  1.  Let  h  =  max  /  (x  ,/>)  and  consider  the  quadratic  programming  problem 
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minimize  p+±dTW  d 

subject  to  Gd-pt  i  ht-f.  (9) 

Theorem  2  For  r  given  by  (6),  let  p  ,d  solve  (9)  defined  at  x.  Then 

(a)  if  p  =  0,  x  solves  (7), 

(b)  if  p  <  0,  d  is  a  descent  direction  for  |/  (x  , .  )||  at  x. 

Proof  By  assumption 

W  =H-ATAD~lA 

is  positive  semi-definite  if  it  is  defined.  The  Kuhn-Tucker  conditions  imply  the  existence  of  a  non- 
negative  vector  such  that 

er|i=  1. 

Wd  +  GT|i  =  0. 

Thus 

d^d  +  dTGTn  =  0. 

so  that  dTGrjt  5  0.  Also 

|tr(he-f-Gd+pe)  =  0, 

implies  h+p~\iTt  =  prGd  S  0,  so  that  p  £  \iTt-h  £  0. 

If  p- 0,  dTW 6  =  0,  so  that  if  6*  0,  there  is  another  solution  p  =  0,  d  =  0,  and  the  Kuhn-Tucker 
conditions  are  then  equivalent  to  (2)  being  satisfied. 

Now  assume  p  <  0.  We  may  write 

//(*) = /  (*4>(x))  y=u,..^ , 

showing  the  dependence  of  tj  on  jc.  Further  for  •pQ  small  enough 

Ilf  (x-^fd, .)«  =  mn/yOH'id). 

Now  define  J  by 


ft  =fj  .  j^J. 


Then  d rV/y  £  p  <  0  ,  j e/,  so  that 

fj  (*+*D  =  fj  (»>+Kdr  Vfj  -KXY2) 

<//(*) 

for  y  >  0  small  enough.  It  follows  that 

II/-  (x-Fjd,  ,)||  <  max/;(x)  =  II/-  (x,  ,)||, 

IS/Sf 

and  the  result  is  proved. 

Remark  This  result  goes  through  for  nonlinear  problems  provided  that  W  is  a  positive  semi-definite 
matrix  (or  is  replaced  by  a  positive  semi-definite  matrix). 

The  new  point  x-Kyd  may  be  obtained  by  a  line  search,  and  it  is  usual  to  choose  y  to  satisfy  the 
inequality 

II f  (*+T|d.  Oil  S  It f  (x,  OIMpx  ,  (10) 

where  x  is  a  small  positive  number,  for  example  0.0001,  and  Y  is  large  in  [0,1],  with  the  value  1  chosen 
if  possible.  Then  the  following  result  may  be  established. 

Theorem  3  Let  {<!*},(/>*}  be  sequences  defined  by  solutions  of  (9)  at  x*  using  (10),  where  the  super¬ 
scripts  k  on  other  quantities  imply  evaluation  at  x*.  Then  if  [W*  }>{<!*}  are  bounded,  the  limit  points  of 
(x* )  solve  (7). 

Proof  The  result  that  (/>k}->  0  may  be  established  by  the  application  of  standard  techniques  available 
in  optimization  theory. 

Going  to  a  subsequence  if  necessary,  let  (x*}->  x*.  Now,  using  boundedness,  and  going  to 
further  subsequences  if  necessary,  p' ,Wk-*W' ,  d*-»  d*  as  Thus  d’,  p'(=0)  solve  (9)  at 
x  =  x*  with  objective  function  value  zero.  Thus  a  (possibly  different)  solution  is  given  by 

d  *  0,  p  =0  and  the  conclusion  that  x  is  a  solution  follows  as  before. 

The  following  algorithm  may  be  interpreted  as  being  of  multiple  exchange  type. 


Algorithm  2 

Step  0  Determine  the  local  maxima  of  /  in  [0,0],  say  t^2 . Jt-  Set  h  =  |(/" (x, .  )||. 

Step  1  Solve  (9),  and  unless  then  is  convergence,  choose  y  so  that  (10)  is  satisfied,  set  x  =  x+yd  and  go 
to  Step  0. 

If  the  problem  (9)  is  solved  with  the  constraints  fixed  as  equalities  ,  then  provided  that  p.  £  0  the 
situation  is  as  before.  However  the  solution  may  now  be  obtained  in  an  efficient  manner  by  steps  simi¬ 
lar  to  those  used  in  Algorithm  1.  An  alternative  exchange  algorithm  may  therefore  be  defined  as  fol¬ 
lows. 

Algorithm  2a 

Step  0  As  in  Algorithm  2. 

Steps  1-5  As  in  Algorithm  1  (with  t,G,  etc.  defined  cm  the  current  set  of  local  maxima  1 \jt2,  -  ■  -  /,), 
except  that  we  can  set  i  =  0  in  Step  4,  and  a  full  step  is  not  necessarily  taken  in  Step  5  for  the  new  x. 

Although  the  steps  of  Algorithm  2a  are  computationally  more  efficient  (and  die  connection  with 
Algorithm  1  immediately  gives  a  desirable  local  property),  there  is  no  guarantee  that  all  the  components 
of  the  multiplier  vector  will  be  non-negative  at  solutions  to  (the  modified)  problem  (9),  in  particular  far 
from  a  solution  to  (1).  Therefore  an  active  set  strategy  (for  handling  negative  p,)  is  required  for  a  prac¬ 
tically  useful  (globally  convergent)  algorithm.  On  the  other  hand,  the  solution  of  the  inequality  con¬ 
strained  problem  (9)  automatically  picks  out  the  correct  active  set  in  the  event  that  not  all  constraints 
are  active.  Eventually,  the  two  algorithms  should  be  effectively  the  same,  so  that  a  second  order  con¬ 
vergent  rate  is  normal  with  Algorithm  2.  An  important  part  of  both  these  algorithms  is  the  calculation 
of  all  the  local  maxima:  this  is  needed  not  just  for  Step  0,  but  also  for  the  implementation  of  the  step 
length  test.  The  process  is  not  a  finite  one,  and  typically  is  achieved  in  two  stages:  firstly  a  grid  search 
to  approximately  identify  the  locations  of  the  maxima;  secondly  a  local  procedure  (for  example 
Newton's  method)  to  give  more  accurate  values.  Incidentally,  this  is  an  area  in  which  the  availability  of 
a  parallel  computing  facility  could  lead  to  great  benefits. 

An  exchange  method  of  either  kind  can  be  applied  to  solve  discretizations  of  continuous  prob¬ 
lems,  with  fairly  obvious  modifications  to  the  algorithms  (reflecting  the  absence  of  (2c)  from  the  condi¬ 
tions  to  be  satisfied):  local  maxima  on  the  finite  set  only  are  used,  W  =  H  in  Step  4  of  Algorithm  1 
and  also  in  (9)  and  the  first  term  on  the  right  hand  side  of  the  system  of  equations  in  Step  4  is  absent. 
However  there  are  potential  difficulties,  if  the  number  of  extrema  of  discrete  and  continuous  problems 


do  not  match.  (In  this  context  extrema  refers  to  points  where  the  norm  is  attained.)  For  example  for 
the  approximation  of  F(w)  =  l/(w-(2+i))  by  a  polynomial  of  degree  3  in  the  first  quadrant  of  the  unit 
circle,  there  are  S  extrema  of  the  continuous  solution,  with  parameter  values  0,  0.259415,  0.733998, 
1.269364,  n/2.  However  the  solution  using  the  algorithm  of  Watson  on  m=101  equispaced  points  gives 
rise  to  6  extrema,  including  the  neighbouring  pair  0.2513  and  0.2670,  and  taking  m  =  1001  equispaced 
points  gives  7  extrema,  including  the  triple  0.73042,  0.73356  and  0.73827.  For  many  problems  this 
discrepancy  does  not  occur,  and  the  extent  to  which  it  can  be  attributed  to  the  use  of  finite  precision 
arithmetic  is  not  clear.  Nevertheless,  there  are  two  possible  dangers  associated  with  this  phenomenon: 
firstly  ill-conditioning  because  of  the  proximity  of  active  points,  and  secondly  failure  of  the  algorithm 
through  failure  to  properly  locate  all  of  these  relevant  local  maxima  as  die  computation  proceeds.  If 
the  situation  is  an  inherent  property  of  the  problem,  it  parallels  a  phenomenon  which  occurs  in  real 
linear  Chebyshev  approximation:  if  a  continuous  real  problem  (with  n  real  coefficients)  is  discretized, 
then  in  the  full  tank  case  there  is  always  a  solution  with  (n+1)  extrema,  and  this  is  the  solution  which  is 
obtained  if  a  standard  technique  is  applied.  However  for  non -Chebyshev  set  problems,  the  solution  to 
the  original  continuous  problem  (assumed  unique)  may  have  fewer  than  (n+1)  extrema:  such  problems 
have  been  referred  to  as  singular  by  Osborne  and  Watson(1969).  A  difference  in  the  complex  case  is 
that  it  does  not  seem  possible  to  predict  in  advance  the  number  of  extrema  of  discretized  problems,  and 
indeed  this  appears  to  depend  on  the  particular  discretization.  Nevertheless  the  same  interpretation  of 
singularity  may  be  made. 

The  above  phenomenon  has  no  bearing  on  the  direct  solution  of  the  continuous  complex  problem 
by  exchange  methods  of  the  type  described  above,  and  Algorithm  2  was  applied  to  a  number  of 
different  problems.  Far  from  the  solution,  convergence  could  be  slow  and  two  modifications  were  found 
to  be  beneficial :  firstly,  W  was  modified  to  W+\tI,  for  some  p2  0,  with  p  being  allowed  to  decrease  as 
the  algorithm  approaches  the  solution;  secondly  in  the  early  stages  only  local  maxima  on  a  discrete  set 
were  determined  (and  so  W  was  chosen  equal  to  H),  with  accurate  local  maxima  obtained  only  when  p 
became  sufficiently  small. 

Example  4  This  is  the  approximation  of  w1  by  a  polynomial  of  degree  2  on  the  first  octant  of  the  unit 
circle.  Initially  local  maxima  were  sought  on  a  grid  of  101  equispaced  points.  The  initial  approximation 
was  given  by  tj  =  (0,0),  j- 23,  *i  =  (1,0)  and  the  performance  of  the  method  is  summarized  in  Table  4. 
The  CPU  time  was  6.63  seconds. 
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An  alternative  to  the  above  method  is  to  revert  to  a  2-phase  approach,  with  algorithm  2  (or 
indeed  Algorithm  2a)  as  a  second  phase.  This  makes  the  2-phase  method  more  robust  (at  the  expense 
of  extra  computation)  and  permits  convergence  to  a  solution  of  (7)  even  if  incorrect  information  is  sup¬ 
plied  at  the  end  of  the  first  phase,  or  poor  approximations  only  are  available.  It  also  means,  however, 
that  while  global  convergence  properties  are  important,  the  main  emphasis  can  be  on  local  behaviour,  if 
Algorithm  2  is  applied  with  all  constraints  initially  set  to  equalities,  then  frequently  no  change  is 
required  to  the  active  set,  so  little  loss  in  efficiency  results  in  using  Algorithm  2  as  opposed  to  Algo¬ 
rithm  2a.  In  addition  W  may  be  used  without  modification,  and  y  =  1  is  the  expected  value  satisfying 
(10).  Therefore  the  next  algorithm  tested  was  based  on  combining  the  method  of  Watson  with  Algo¬ 
rithm  2.  In  fact  a  version  of  the  first  method  was  used  where  solutions  were  obtained  on  successive 
discrete  sets  defined  as  follows.  Initially  2a +1  equispaced  points  were  used.  Subsequently  each  set  con¬ 
sisted  of  the  current  set  of  extreme  points  supplemented  by  all  the  local  maxima  of  /  on  the  original 
discrete  point  set.  When  a  sufficiently  small  value  of  p  was  reached,  or  a  sufficiently  small  increment 
vector  was  obtained.  Algorithm  2  was  entered. 

Algorithm  3 

First  Phase  The  algorithm  of  Watson,  modified  as  described  in  the  previous  paragraph. 

Second  Phase  Algorithm  2,  with  all  constraints  of  (9)  initially  set  to  active. 
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Both  phases  of  this  algorithm  involve  the  solution  of  a  sequence  of  quadratic  programming  prob¬ 
lems,  so  that  in  addition  to  giving  a  desirable  combination  of  global  and  rapid  local  convergence,  they 


also  make  use  of  the  same  software.  At  each  iteration  of  both  phases,  a  particular  subset  of  discrete 
points  from  [a,(5]  is  identified  and  the  constraints  of  the  quadratic  programming  problem  are  defined  by 
this  subsec  in  the  first  phase,  this  remains  constant  for  several  iterations,  and  not  all  constraints  are  nor¬ 
mally  active;  in  the  second  phase,  the  subset  changes  at  each  iteration,  and  all  constraints  are  usually 
active. 

Consider  Example  4  solved  by  Algorithm  3  (from  the  same  initial  approximation).  Table  5  gives 
results  of  the  application  of  the  above  algorithm  applied  to  the  discretized  problem.  The  number  k 
gives  the  outer  iteration  count  ( the  number  of  times  a  discrete  solution  is  obtained  on  a  fixed  discrete 
subset  of  the  original  discrete  set),  q  is  the  number  of  quadratic  programming  subproblems  solved  at 
each  outer  iteration,  ||r*||  is  the  value  reached  of  the  norm  on  the  current  discrete  set,  and  ||r||  is  the 
norm  evaluated  on  all  101  points.  The  number  l  is  the  number  of  local  maxima  added  in  at  each  step, 
and  p  denotes  the  value  of  p  reached  on  termination.  The  data  input  to  Algorithm  2  was: 

z  =  {(0.3679780,0.8883755),  (-1.988566,-1.988564),  (2.630987,1.089790)}. 

After  2  iterations,  the  maximum  modulus  component  of  the  increment  vector  was  less  than  0.00001  and 
the  solution  obtained  was 

z  =  {  (0.368115,0.888709),  (-1.989059,-1.989059),  (2.631323.1.089929)}, 
t*  =  (0,0. 198232,0.5871 65,i), 

X'  =  (0.167767,0.332231,0.332236,0.167765), 
with  ||r*  ||  =  0.147077  .  The  total  CPU  time  was  7.98  seconds. 


k 

<1 

Hr*  II 

llrll 

/ 

P 

1 

14 

0.011666 

0.022209 

3 

-2.0  -7 

2 

4 

0.014607 

0.014865 

2 

1 

i 

o 

3 

3 

0.014706 

-4.6  -9 

Table  5 

For  approximation  problems  defined  on  the  whole  of  the  unit  circle,  it  is  not  uncommon  for  there 
to  be  a  constant  error  curve,  in  other  words  for  the  minimum  norm  value  to  be  attained  at  all  points  on 


the  unit  circle.  Studies  of  this  phenomenon  for  polynomial  (and  rational)  approximation  have  been  made 
by,  for  example,  Trefethen  (1981a,1981b).  In  this  case  it  is  clear  that  Algorithm  2  will  eventually  break 
down;  on  the  other  hand  ,  for  methods  of  two-phase  type,  it  is  easy  to  identify  this  situation  at  the  con¬ 
clusion  of  the  first  phase. 

Example  5  Let  f  (w)  =  l/(w-2)  be  approximated  by  a  polynomial  of  degree  6  on  the  whole  of  the  unit 
circle.  The  performance  of  the  algorithm  of  Watson(1988)  applied  on  100  equispaced  points  from  z  =  ei 
is  summarised  in  Table  6.  At  the  end  of  the  second  iteration  ||r||  =  0.005208  attained  at  all  100  points, 
and  the  conclusion  that  the  error  curve  is  a  constant  on  the  unit  circle  follows. 
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The  possibility  of  this  kind  of  behaviour  points  the  way  to  another  source  of  difficulty  in  accu¬ 
rately  computing  solutions  to  continuous  complex  Chebyshev  approximation  problems:  although  the 
error  curve  is  not  constant,  it  may  be  close  to  being  constant  This  means  that  identification  of  the  posi¬ 
tions  of  the  points  where  the  norm  is  attained  may  be  quite  difficult  and  the  positions  of  the  local  max¬ 
ima  as  the  computation  proceeds  will  be  very  sensitive  to  changes  in  the  coefficient  values. 

1 

Example  6  Consider  the  approximation  of  /(w)  =  1/(1+ (w+1)2)2  by  ]£zy(l+>v)1->  on 

/-i 

[w  :w  =  iy -UC&y  <20) .  This  is  an  another  example  of  a  complex  problem  which  has  a  solution  for 
which  z  is  real,  and  advantage  can  be  taken  of  this  to  reduce  the  size  of  the  computations:  in  particular 
symmetry  can  be  used  to  restrict  consideration  to  non-negative  values  of  y.  In  Table  7a  is  shown  the 
result  of  applying  the  algorithm  as  before  on  51  equispaced  points  on  0£y£20,  taking  n  =  4  and  start¬ 
ing  from  z  =  *j.  The  minimum  value  of  ||r||  is  0.016046  attained  at  the  5  points  defined  by 
t  =  (0.4,0.8,1.6,2.0,20.0)r.  The  points  corresponding  to  local  minima  of  I/-;  I  are  0.0,  12  and  3.6  with 
values  0.015983,  0.016036  and  0.016039  respectively.  Table  7b  shows  the  performance  of  Algorithm  2 
applied  from  the  discrete  solution;  the  continuous  problem  has  just  3  extrema  and  the  sensitivity  of  the 


local  maxima  to  changes  in  the  coefficients  is  clearly  demonstrated.  The  total  CPU  time  was  7.81 
seconds. 
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Finally  the  performance  of  Algorithm  3  applied  to  some  other  examples  is  summarized  in  Table 
8.  The  different  functions  used  were 

X 

/i(w)  =  (sinw/w)2, 

f^w)  =  (w- (2+Or1, 

i 

/j(w)  =  (l+(w+l)a)  J, 

and  other  information  defining  the  problems  being  solved  is  given  in  the  table.  All  initial 


i 


approximations  were  z  =  ei.  The  number  8  gives  the  maximum  modulus  component  of  the  increment 
vector  on  termination,  p  denotes  the  final  value  of  p ,  ||r||  is  the  final  value  of  the  norm,  and  CPU 
denotes  the  CPU  time  in  seconds. 
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4.  Concluding  remarks 

Some  numerical  methods  have  been  presented  for  solving  complex  Chebyshev  approximation  problems 
defined  on  a  contour  of  the  complex  plane.  It  has  been  assumed  that  a  one-dimensional  parameterization 
of  the  contour  is  available,  and  this  has  been  exploited  to  increase  efficiency  of  the  methods,  although  it 
is  not  a  limiting  factor.  Numerical  results  have  been  presented  to  demonstrate  the  effectiveness  of 
different  approaches  for  a  variety  of  linear  problems,  and  examples  have  been  used  to  illustrate  various 
points.  The  emphasis  has  been  on  the  provision  of  methods  which  can  combine  global  convergence  with 
satisfactory  rapid  local  convergence  and  it  seems  that  this  is  best  achieved  within  the  context  of  a  two- 
phase  approach:  this  is  characterized  by  the  use  of  the  solution  to  a  discretization  of  the  original  prob¬ 
lem  (and  other  information  produced)  as  input  to  a  method  with  (primarily)  good  local  properties  for 
satisfying  the  characterization  conditions.  Conventional  methods  of  this  type  do  not  monitor  progress  in 
the  second  phase  (and  in  particular  are  not  descent  methods),  and  rely  on  the  input  data  being 
sufficiently  good  that  convergence  to  a  solution  of  the  original  problem  is  achieved  without  difficulty. 


The  intention  here  has  been  to  provide  a  measure  of  satisfactory  progress  in  the  second  phase,  so  that 
(at  the  expense  of  additional  computation)  convergence  to  the  solution  can  be  more  reasonably 
guaranteed.  While  the  numerical  experience  is  still  somewhat  limited,  it  is  hoped  that  the  present  work 
will  assist  in  the  quest,  and  provide  a  satisfactory  framework,  for  good  numerical  methods  for  solving 
linear  complex  Chebyshev  approximation  problems. 

Finally,  it  is  clear  that  nonlinear  problems  may  be  tackled  by  the  methods  presented  here.  The 
second  phase  (in  theory)  presents  no  difficulty,  and  so  the  main  requirement  is  for  a  satisfactory  method 
for  solving  the  discretized  problem,  and  this  involves,  in  particular,  deciding  how  to  deal  with  the  Hes¬ 
sian  matrix  of  / .  One  possibility  is  to  ignore  that  part  of  the  Hessian  matrix  which  requires  second 
derivatives  of  r ,  (the  second  term  on  the  right  hand  side  of  (4))  leading  to  a  method  of  Gauss-Newton 
type.  While  this  is  simple,  it  is  unlikely  to  be  generally  effective,  and  quasi-Newton  or  finite  difference 
approximations  to  the  Hessian  matrix  are  mare  attractive  propositions.  It  may  be  that  there  is  little  to 
be  gained  by  considering  alternatives  to  standard  methods  for  nonlinear  minimax  problems,  particularly 
if  they  are  tailored  to  exploit  special  structure  and  properties  as  is  done  by  Yuying  Li(1988).  This 
remains  to  be  investigated  and  will  be  the  subject  of  future  research. 
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A  FAST  ALGORITHM  FOR  LINEAR  COMPLEX  CHEBYSHEV 
APPROXIMATION 
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Abstract  A  natural  generalization  of  the  Remez  algorithm  from  real  approximation  to 
complex  is  presented.  The  algorithm  is  the  first  quadratically  convergent  of  its  kind. 

Keywords  Chebyshev  approximation,  Remez  algorithm,  quadratic  convergence. 

1  Introduction 

Given  a  complex  function  F  analytic  on  a  specified  domain  in  the  complex  plane,  how 
can  one  construct  the  polynomial,  of  a  prescribed  degree,  that  best  approximates  F  in 
the  Chebyshev  sense?  Applications  for  such  an  algorithm  can  be  found  in  [2],  [3],  [5],  [6], 
and  [8].  In  the  past,  complex  Chebyshev  polynomial  approximation  has  been  far  less  well 
understood  than  its  real  analogue.  In  particular,  the  quadratically  convergent  Remez 
algorithm  ([7]  and  [13])  for  real  approximation  has  not  been  satisfactorily  generalized 
to  a  quadratically  convergent  algorithm  for  complex  approximation.  In  this  paper,  we 
describe  a  natural  generalization  of  the  real  Remez  algorithm  to  complex  that  converges 
quadratically  when  conditions  similar  to  those  for  the  real  case  are  satisfied. 

The  organization  of  this  paper  is  as  follows.  In  Section  2  we  formulate  our  approx¬ 
imation  problem  in  a  form  that  facilitates  the  generalization.  In  Section  3,  we  take  a 
short  digression  on  the  real  Remez  algorithm  before  the  complex  Remez  algorithm  is 
presented.  The  convergence  properties  of  the  algorithm,  both  global  and  local,  are  dis¬ 
cussed  in  Section  4.  Although  the  assumptions  in  Section  4  for  quadratic  convergence 
are  not  met  very  often  in  practice,  we  are  able  to  relax  them  by  an  extension  of  the 
algorithm  in  Section  5.  Finally,  some  concluding  remarks  are  made  in  Section  6. 

2  Formulation 

Consider  a  complex-valued  function  F  analytic  in  a  domain  enclosed  by  a  smooth  closed 
curve  in  the  complex  plane.  Let  7  :  [0, 1]  — ►  C,  7(0)  =  7(1),  be  a  smooth  parametriza- 
tion  of  that  curve,  which  is  the  boundary  of  the  domain.  Define  /,  y>i>  •  •  •  ,v?2n,  and 


p  as  follows: 


F(t)  :=  /( 7(0), 

<pi(t)  :=  R  e(V-1(0).  /=l,2,...,n 

¥>»+/(*)  :=  tlm(7'_1(0)i  l  =  l,2,...,n 

3n 

p(A,«)  :=  AiV>l(<)  for  any  A  =  [Alt  A2, . . . ,  A2n]T  G  R2n. 

1=1 

Let  ||  •  ||  be  the  Chebyshev  norm  over  [0,1]-  Then  the  complex  Chebyshev  approximation 
problem  is  to  find  A*  G  R  and  A*  G  R2n  such  that 

h*  =  || /  -  J* A*,  -)||  <  ||/  -  P(A,  -)||  for  all  A  G  R2". 


3  A  Complex  Remez  Algorithm 

In  this  section,  we  fLst  revisit  the  familiar  real  Remez  algorithm  and  view  it  as  a  simplex 
algorithm  that  solves  the  dual  of  the  approximation  problem.  Although  that  dual  is 
a  semi-infinite  linear  program,  the  method  of  solution  needs  no  discretization  or  two 
different  phases,  as  done  in  [4]  and  [10].  Once  the  real  Remez  is  realized  as  such,  simply 
doubling  its  dimension  yields  an  algorithm  for  the  complex  problem. 


3.1  The  real  Remez  algorithm 

For  the  duration  of  this  subsection,  assume  that  /  is  real  and  7  =  1-  Thus  F  —  f  and 
7 i(t)  =  =  1,2, ...,n.  The  approximation  problem  can  be  viewed  as  the  following 

minimization  problem:  Determine  hm  €  R  and  A*  G  R"  so  as  to  minimize  h,  subject  to 

h  >  e~l*  (/(t)  -  p( A,  t)) ,  for  all  (t,  d)  G  [0, 1]  X  {0,  *}. 

The  dual  [9]  of  the  minimization  problem  is  as  follows:  Determine  an  irfinite  vector 
r  :  I  — *  [0, 1],  where  I  :=  [0, 1]  X  {0, x),  so  as  to  maximize 

*=  E 

(t.S)C? 


subject  to 

E  r(M)  =  l. 

M)€I 

53  r(t,d)e-‘V/(0  =  0,  /=  l,2,...,n. 

The  infinite  sum  can  be  justified  since  the  optimum  is  achievable  by  a  vector  r*  with 
at  most  n+1  nonzero  entries. 


Given  t  =  [<i,t2,  •  •  •  ,*n+i]T  €  [0,l]n+1  and  d  =  [t?x,  t?2, . . . ,  t?„+x]T  6  (0,x}n+1, 

define  two  n  +  1-vectors  and  an  (n  +  1)  X  (n  +  1)  matrix  as  follows: 

6  :=  [1,0,0,.  ..,0]r 

a(*,d)  :=  [1  ,e-l'}<?l(t),e~l*V2(t),...,e~l'}<pn(t)\T 
A(t,#)  :=  [a(ti,  t?x ),a(t2,  t?2), . . .  ,a(<n+i , &n+i )]. 

The  one-point  exchange  Remez  algorithm  for  real  approximation  can  be  stated  as  fol¬ 
lows: 

Algorithm  1 

Step  0  Find  an  initial  t  €  [0,  l]n+1  and  €  {0,ir}n+1  such  that  A(t,ti)  is  nonsingular 
and  A-1(t,#)  b  >  o. 

Step  1  Define  r(t,t?)  :=  A-1(t,i9)  •  6.  From  (t,#)  determine  the  unique  cr  :=  ^ 

such  that 

e  —  =  fi,  j  —  1,2, . . . ,n  +  1. 

Step  2  Determine  t'  such  that  |/(t')  -  p(A,l')i  =  ||/  -  p(A,-)||.  Define 

^  :=  (  0  if  f(?)  -  p(A,  t')  >  0; 
l  ic  otherwise. 

Since  h  <  \\f  -  p(A*,-)||  <  ||/(t')  -  p(A,t')l|,  whenever  ||/(t')  -  p(A,t')||  -  A  is 
small,  h  and  A  are  good  approximations  to  the  optimal  solution  and  we  terminate 
the  algorithm.  Otherwise,  we  move  on. 

Step  3  Exchange  with  one  of  the  («>,i?>)’s,  j  =  l,2,...,n+  1.  This  exchange 

can  be  determined  by  solving  the  following  small  linear  programming  problem  (via 
the  simplex  algorithm):  Determine 


»•(*!,  i?i ),  r{t2,  d2), . . . ,  r(tn+x,  t?n+i ),  r(t',  tf')  6  [0, 1] 


so  as  to  maximize  the  inner  product 


subject  to 

(A(M)  o(t',tH]r  =  d. 

The  optimal  basis  of  this  problem  is  exactly  {(fj ,  tfx), (t2, i?2), . . . , (tn+1,  t^n+l )} 
with  one  of  the  elements  replaced  by  (<',«?').  Rename  the  new  basis  as  (t,0).  Go 
back  to  Step  1. 


3.2  The  complex  Remez  algorithm 

We  are  now  ready  to  go  back  to  the  complex  approximation  problem,  which  can  be 
stated  as  the  following  minimization  problem:  Determine  h”  G  R  and  A*  €  R2n  so  as 
to  minimize  h ,  subject  to 

h  >  Ke(e~^lf(t)  -  p(A,t)]),  for  all  6  [0,1]  x  [0,2*]. 

The  dual  of  the  minimization  problem  is  as  follows:  Determine  an  infinite  vector  r  : 
1  -»  [0,1],  where  1  :=  [0,1]  X  [0,2*],  so  as  to  maximize 

*=  E  Kt,i?)Re(e-‘V(0) 

(t.t»)er 

subject  to 

r(M)=l>  and 

r(t,t?)Re(e_l,,vJJ(t))  =  0,  /  =  1,2, . . .  ,2n. 

The  infinite  sum  can  be  justified  since  the  optimum  is  achievable  by  a  vector  r*  with 
at  most  2n  +  1  nonzero  entries. 

Given  t  =  [ti,<3,  •  •  •  ,<2„+i]T  €  [0,  l]n+1  and  . tfzu+if  €  [0,2*]2"+1, 

define  two  2n  +  1-vectors  and  an  (2n  +  1)  X  (2n  +  1)  matrix  as  follows: 

b  :=  [1,0,0,...,0]T 

«(*,*)  :=  [l.R-e(c'“,S5i(0),Re(e-‘V2(<)),---,Re(e-‘,,v>2„(t))]T 

:=  [a(<i,1?i),a(t2,^2),  •  •  •  ,<*(<2»+l>  l?2«+l)]. 

The  one-point  exchange  Remez  algorithm  for  complex  approximation  is  merely  a  straight¬ 
forward  mimic  of  Algorithm  1. 

Algorithm  2 

Step  0  Find  an  initial  t  €  [0,  l]2n+1  and  t?  e  [0,2*]2n+1  such  that  A(t,tf)  is  nonsingular 
and  •  b  >  o. 


Step  1  Define  r(t,0)  :=  •  b. 

such  that 


From  (t,  0)  determine  the  unique  a  := 


Re(e-‘*[/(ti)- **,!;)])  =  A, 


j  =  l,2,...,2n  +  1. 


Step  2  Determine  «'  such  that  |/(t')  -  p(A,t')l  =  ||/ -  p(A,  )||.  Define  the  angle 
&  Arg (f(f)  —  p(A,t')).  If  ||/(t')  —  p(A,  t/)||  —  h  is  small,  h  and  A  are  good 
approximations  to  the  optimal  solution  and  we  terminate  the  algorithm.  Other¬ 
wise,  we  move  on. 

Step  3  Exchange  with  one  of  the  j  =  1,2, . . .  ,2n  +  1  as  done  in  the 

previous  algorithm  for  the  real  case.  Go  back  to  Step  1. 


4  Convergence  of  the  Algorithm 

The  Remez  algorithm,  whether  real  or  complex,  generates  a  sequence  of  iterates  r^k\ 
(*<*>, t?(*>),  and  =  [  |  ■  Questions  of  convergence  are  naturally  centered  on  the 


quantity 


Does  rfk  converge  to  0,  and,  if  so,  how  fast?  We  answer  these  questions  below  in  order. 
4.1  Global  convergence 

In  the  discussion  of  the  real  Remez  algorithm,  [7]  shows  that  tfc  — »  0.  The  proof  is  a 
direct  consequence  of  the  fact  that  >  A  >  0  for  some  uniform  lower  bound  A  for  all 
the  r(*).  Such  a  bound  does  not  exist  in  general,  however,  for  two  (or  higher)  dimension 
real  approximation  or  complex  approximation.  Nevertheless,  similar  to  the  situation  in 
the  simplex  algorithm,  it  is  reasonable  to  assume  rW  >  o  for  all  k. 

Theorem  1  If  >  o  for  k  =  1,2,3, . . .,  then 

lim  inf  rjk  =  0. 

k—oo 

Proof  A  complete  proof  can  be  found  in  [11].  | 

In  practice,  lim  inf *_<„  r?*  =  0  means  that  the  algorithm  terminates  in  a  finite  num¬ 
ber  of  steps  for  any  positive  stopping  criterion. 


4.2  Local  convergence 

For  the  real  Remez  algorithm,  it  is  proved  in  [7]  that  the  sequence  {%}  is  majorized  by 
a  sequence  {£*},  f>k  >  Vk  for  all  k,  that  converges  to  zero  quadratically.  The  standard 
assumption  ([1],[7],  and  [i  3])  for  real  approximation  is  that  the  optimal  /  -  p  has  n  +  1 
extrema  (alternations)  at  which  the  second  derivatives  are  nonzero.  The  assumption 
and  convergence  result  are  generalized  to  complex  approximation  in  the  next  theorem. 

Theorem  2  Let  the  function  \  f(t)  -  p(A*,t)|  have  exactly  2n+  1  extrema  t,*,  1  <  j  < 
2n  +  1,  and  let  r*  :=  A-1(t*,0*)-6  >  o,  where  dj*  :=  Arg(/(tj*)-p(A*,tj*)) ,  1  <  j  < 
2n  +  1.  Furthermore,  let  the  second  derivatives  with  respect  to  t  of  \  f(t)  —  p(A*,t)  |  be 
nonzero  at  each  of  the  2n  +  1  extrema.  Then  there  exists  a  sequence  for 

all  k,  and  two  constants  M  and  K  such  that 

hn+1+k  <  Mil  for  k>  K. 

Proof  The  complete  proof  is  found  in  [11].  I 

Unfortunately,  The  proof  in  [11]  is  too  long  to  be  included  here.  That  length, 
however,  is  mainly  due  to  the  fact  that  the  algorithm  is  a  one-point  exchange  instead 
of  a  multipie-exchange  one.  In  the  rest  of  this  section,  we  first  explain  why  quadratic 
convergence  is  obtainable.  Then,  we  present  a  multiple  exchange  version  of  the  Remez 
algorithm.  Although  this  version  works  only  when  the  iterate  is  close  enough  to  the 


optimum,  the  proof  of  its  quadratic  convergence  is  sufficiently  short  to  be  presented 
here. 

Consider  the  nonlinear  system  of  equations 

Re(e-“,>[/(tj)-p(A,tj)]) -h  =  0  j  =  1,2, . . .  ,2n  +  1 

(4.1)  =  0  j  =  l,2,...,2n+l 

Im(e-‘^[/(ti)-p(A,tj)])  =  0  j  =  l,2 . 2#  +  l 

Denote  the  3(2n + 1)  real  parameters  by  groups  of  2n  + 1  thus  x  (a,  t,  ■d).  Moreoever, 

denote  the  three  groups  of  2n  +  1  lefthand  sides  by  gj(x),j  =  1,2,3.  Finally,  define 
G(x)  :=  [pf(*),02(*),0^(x)].  Suppose  the  assumptions  of  Theorem  2  are  satisfied; 
then  it  is  not  hard  to  show  that  x*  :=  (<r*,t*,i?*)  is  an  isolated  zero  of  Equation  4.1. 
Moreover,  we  can  prove  the  next  theorem. 

Theorem  3  With  the  assumptions  of  Theorem  2,  the  Jacobian  J(x*)  of  G  at  x*  is 
invertible. 

Proof  Partition  the  Jacobian  into  3x3  blocks  of  dimension  2n  + 1  each.  Because  of  the 
structure  of  G ,  the  derivatives  with  respect  to  t  and  •&  are  all  diagonal  matrices.  More¬ 
over,  because  t*  and  are  extrema  of  Re(e“‘^[/(t)  -  p(A*,f)])  and  \f(t)  -  p(A*,t)|, 

(9i)t(*)  =  (gi)fl(*)  =  «  for  all  x, 

(*a)t(*")  =  diag(^l/(0-p(A*,t)|  L,-..,^|/(«)-p(A*,t)|  I  ), 

.  '*»  *2>»+l 

(SaM**)  =  diag(-A*, -h‘). 

Finally,  =  —AT(tm, i3*).  Clearly,  the  Jacobian  at  x  is  block  lower  triangular 

with  the  diagonal  blocks  invertible.  | 

It  is  easy  to  check  that  the  Jacobian  is  Lipschitz  continuous  in  x  and  thus  Newton 
iteration  would  converge  quadratically.  This  approach  is  not  taken  in  real  approxima¬ 
tion,  however,  because  of  its  high  cost  since  the  Newton’s  approach  is  of  dimension 
2(n  +  1)  instead  of  n  +  1,  the  dimension  of  Remez.  The  situation  is  similar  in  complex 
approximation.  Indeed,  we  can  show  that  the  following  multiple  exchange  complex  Re¬ 
mez  algorithm,  of  dimension  2n  + 1,  converges  as  fast  as  Newton  iteration  does,  which  is 
of  dimension  3(2n+  1).  The  multiple  exchange  complex  Remez  algorithm  is  Algorithm 
2  with  Steps  2  and  3  replaced  as  follows: 

Step  2/  Determine  ^ ,  t'2, . . . ,  f2n+i  near  ti ,  t2, . . . ,  t2n+1 ,  respectively,  such  that  the  first 
derivatives  of  |/(t)  -  p{ A,  t)|  with  respect  to  t  vanish.  Define  as  the  arguments 
of/^J-lKA^). 

Step  3'  Replace  (t,  0)  by 

This  algorithm  works  when  (t,  )  is  dose  to  the  optimum  because  A~1(V,  0m)  b  >  a 
implies  A“‘(t,^)  •  b  >  o  whenever  (t,tf)  is  dose  to  (t*,tf*). 

Now,  to  show  the  equivalent  convergence  rate  of  the  Newton  and  Remez  iterations, 
we  consider  the  updates  x^  and  x^  after  one  iteration  of  the  Newton  and  the  Remez 


algorithms,  respectively,  both  starting  at  a  common  point  x  very  close  to  the  optimum 
x*,  ||x  —  x*||  <  e.  We  will  show  that  ||x^  -  xlwl||  <  Me2  for  some  constant  M 
independent  of  e. 

First,  the  Newton  update  x^  is  defined  by 
(4.2)  J(x)  •  (x  -  x<-*0)  =  G(x). 

Next,  consider  The  vector  <7^  is  defined  by 

(9i)a(x)  •  {<*  ~  )  =  0i(*)- 

Based  on  crW,  t W  and  ‘d^  are  determined  by 

=  S3(*(W>)  =  o- 

Thus, 

.03(*)J  [s3(*)J  dx  Ls3(x)J  y' 

where  ||y||  <  M\£2  for  some  M\  indepedent  of  £.  Thus,  x^  can  be  characterized  as 


(4.3) 


( 


J(*)- 


0  (ffi)t(*)  (Si)t?(*)l 

00  o 

00  o 


^  •  (x  -  X^l)  = 


=  G(x)  +  y. 


Subtracting  Equation  4.3  from  Equation  4.2  gives 
(4.4)  J(x)  •  (x(*)  —  x^)  =  z 

where  ||z||  <  M^e2  for  some  Afj  independent  of  e.  This  is  because  ||(9i)t(x)||,  ||(ffi)^(x)||, 
x  —  x(^)||  <  M3s  for  some  Af3  independent  of  e.  Finally,  J(x*)  is  invertible,  and 
|  J-1(x)||  is  uniformly  bounded  near  x*  and  the  proof  is  complete. 

Numerical  examples  illustrating  the  convergence  behavior  can  be  found  in  [12]  and 


and  | 
thus 


[11]- 


5  An  Extension  of  the  Complex  Remez  Algorithm 

In  our  experience  ([11]  and  [12]),  whenever  the  assumptions  in  Theorem  2  are  not 
satisfied,  the  number  of  extrema  is  insufficient.  Fortunately,  in  that  case  we  can  extend 
the  algorithm  slightly  to  restore  quadratic  convergence.  Details  of  this  extension  are 
presented  in  [11]  and  [12];  the  idea  is  roughly  as  follows.  If  the  number  of  extrema 
is  d  fewer  than  the  ideal  number,  one  can  define  a  function  u  :  whose 

zero  corresponds  to  the  solution  of  the  original  approximation  problem.  Moreover, 
to  evaluate  the  function  and  its  Jacobian,  we  need  only  to  apply  the  complex  Remez 
algorithm  to  a  problem  that  satisfies  all  the  conditions  needed  for  fast  convergence. 
Consequently,  Newton  iteration  applied  on  tt  is  efficient.  The  quadratic  convergence  of 
this  extended  algorithm  is  fully  analyzed  in  [11]. 


6  Conclusion 


We  have  shown  that  linear  complex  Chebyshev  approximation  can  be  naturally  viewed 
as  a  special  case  of  two-dimensional  real  approximation  for  which  the  Remez  exchange 
algorithm  converges  quadratically.  We  believe  such  a  view  can  help  our  understanding 
in  other  aspects  of  complex  approximation  such  as  Chehyshev  approximation  by  rational 
functions. 
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